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PREFACE 
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INTRODUCTION 


This  final  report  describes  all  efforts  performed  under  Air  Force 
Geophysics  Laboratory  (AFGL)  contract  number  F19628-86-C-0033,  entitled 
"Evaluation  of  the  Feasibility  of  Applying  Artificial  Intelligence  Techniques 
to  the  Prediction  of  Visibility  at  Selected  DoD  Bases  and  Development  of  a 
Knowledge-Based  Expert  System." 

The  format  of  the  final  report  is  to  first  introduce  the  reader  to 
the  project  objectives  and  then  proceed  with  a  general  discussion  on  Artificial 
Intelligence/Knowledge-Based  Expert  Systems  (AI/KBES).  Next,  we  follow 
the  evolution  of  the  KBES  through  various  stages  by  first  introducing  the 
reader  to  a  general  discussion  of  each  generic  step  and  then  specifically 
applying  the  discussion  to  our  KBES  development.  An  expert  system  was 
developed  for  three  airbases  including  Seymour  Johnson,  North  Carolina, 

Dover,  Delaware,  and  Fort  Bragg,  North  Carolina.  The  construction  of  each 
system  is  similar  in  terms  of  physical  principles  such  that  the  three  expert 
systems  are  variations  of  each  other.  We  call  the  system  "Zeus."  The 
results  of  evaluating  each  system  using  independent  data  for  each  airbase  are 
presented  and  the  lessons  learned  are  discussed.  Finally,  we  present  our 
conclusions  as  well  as  recommendations  for  future  work. 

1.1  AFGL  PROJECT  OBJECTIVES 


The  project  objectives  were  twofold: 

1.  To  determine  whether  forecast  techniques  can  be 
expressed  using  AI  structures  and  software 

2.  To  determine  whether  Air  Weather  Service  (AWS) 
base  personnel  would  accept  the  idea  of  a  KBES 
providing  meteorological  advice. 

This  effort  is  a  proof-of-concept  and  has  never  been  tried  within  AWS.  Thus, 
many  of  the  technical  AI  terms  and  the  software  were  unknown  at  the  start  by 
AWS  base  personnel. 

Longer  term  objectives  are  to: 

1.  Improve  short-range  (0-  to  12-h)  Terminal 
Aerodrome  Forecasts  ( TAF ) 

2.  Improve  forecaster  performance  in  relation  to 
use  of  time  and  focusing  of  resources 

3.  Decrease  orientation  period  (training)  time 
in  a  new  location 


4.  Increase  safety  through  more  accurate  predictions. 

These  longer  term  objectives  are  the  goal  of  an  integrated  AI  effort  within  AWS. 


1.2  ARTIFICIAL  INTELLIGENCE/EXPERT  SYSTEMS-A  BRIEF  OVERVIEW 

Artificial  Intelligence  Is  the  science  of  understanding  how  to  make 
machines  of  any  type  create  the  results  that  would  normally  require  human 
intelligence. 

Thus,  AI  is  a  subfield  of  computer  science  that  is  concerned  with 
various  concepts  and  methods  of  symbolic  inference  and  symbolic  representation 
of  knowledge  to  make  the  inferences. 

AI  generally  Involves  the  investigation  of  two  broad  topic  areas: 

1.  Intelligent  thought  and  action  Itself 

2.  Computer  software  and  hardware  to  express 
what  is  understood  about  Intelligence. 

More  specifically,  the  evolution  of  AI,  as  shown  in  Fig.  1,  has  led  to  three  major 
areas  of  emphasis: 

•  Natural  language  processing 

•  Robotics 

•  Expert  systems. 

The  first  area  involves  computer  processing  in  the  English  language 
itself,  such  as  speech  Interfaces  that  are  used  for  several  DoD  aircraft 
control  projects.  The  user  speaks  to  the  computer  and  the  computer  responds 
by  initiating  an  action  or  by  talking  with  the  user. 

The  second  area  involves  mechanical  devices  designed  to  Improve  either 
the  efficiency  of  an  operation  or  to  perform  under  hazardous  conditions.  For 
example,  robotic  systems  are  used  for  such  things  as  retrieval  of  engine 
parts  from  storage  areas  to  demilitarization  of  chemical  warfare  munitions. 

The  expert  system  area  has  been  shown  to  have  Increasing  promise  for 
use  in  environmental  decision  making  (1)  and  in  meteorology  (2,  3). 

A  KBES  (see  Fig.  2)  is  basically  a  structured  collection  of  knowledge 
that  can  Interact  with  users.  This  Interaction  Is  accomplished  by  a  series 
of  questions  and  answers  or  by  a  series  of  data  inputs  directly  into  the 
system.  These  questions  or  data  Inputs  are  in  a  controlled  sequence  that  is 
designed  to  access  the  knowledge  data  base.  The  end  product  results  in  the 
user  receiving  recommendations  with  a  probability  of  success.  Expert  systems 
also  include  the  capability  to  describe  their  line  of  reasoning  and  to  play 
"what  if"  games  with  various  data  input. 
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Figure  1.  Evolution  of  AI. 
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TABLE  1  illustrates  the  basic  differences  between  conventional  and 
symbolic  programming.  The  major  differences  include  numerically  addressable 
data  versus  symbolically  structured  data  and  interactive  explanation  features. 


TABLE  1.  Basic  Difference  Between  Conventional  and 
Symbolic  Programming. 


Conventional 

Programming 

Symbolic 

Programming 

Oriented  toward 
numerical  processing 

Oriented  toward 
symbolic  processing 

Numerically  addresses 
data  base 

Symbolically  structured 
knowledge  base 

Algorithms 

Declarative 

knowledge 

Sequential,  batch 
processing 

Highly  interactive 
processing 

Program  specification 

Iterative  refinement 

Mid-run  explanation 
impossible 

Mid-run  explanation 
easy 

A  meteorological  expert  system  would  differ  from  more  traditional 
Model  Output  Statistics  (MOS)  or  numerical -model -related,  output-oriented 
programs  in  three  key  areas: 

•  Representation  of  information  (data/knowledge) 

•  Processing 

•  Explanation. 

The  algorithmic  approach  to  weather  forecasting  is  basically  an 
approach  that  if  given  the  correct  input  data  you  will  get  a  correct  answer. 

Of  course,  part  of  the  problem  in  numerical  weather  predicition  (NWP)  is  the 
right  initial ization  of  data  (a  subject  that  is  always  discussed  on  the 
human-machine  mix  product).  NWP  models  manipulate  data  only.  The  KBES,  on  the 
other  hand,  deals  with  heuristics  and  knowledge  (along  with  data  needed  to 
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use  the  knowledge).  Expert  systems  are  able  to  represent  the  total  meteoro¬ 
logical  picture  using  rules  of  thumb  or  structures  and  are  not  encumbered  by 
number  crunching. 

A  second  difference  involves  processing.  MOS,  for  example,  relates 
meteorological  categories  and  key  parameters  to  statistical  analysis  based  on 
a  comprehensive  data  base.  MOS  is  oriented  toward  numerical  and  statistical 
processing.  Expert  systems,  in  contrast,  are  oriented  toward  symbolic 
processing  and  inferential  reasoning.  The  expert  system  deals  with  manipulatory 
knowledge  and  not  statistical  regression  equations. 

A  third  area  of  difference  is  in  the  explanation  function.  MOS 
cannot  explain  why,  for  example,  it  is  changing  the  temperature  in  such  a 
manner.  Many  forecasters  are  wary  of  MOS  when  a  front  is  forecast  to  come 
through  at  1200  Z.  Forecasters  are  also  cautious  in  using  numerical  output 
during  seasonal  changes.  Many  numerical  model  discrepancies  have  been 
pointed  out  in  the  daily  human-machine  mix  weather  discussions.  Expert 
systems,  in  comparison,  have  explanatory  facilities  and  the  ability  to 
explain  both  their  line  of  reasoning  and  individual  rules.  An  expert  system 
knowledge  data  base  also  has  the  capability  to  be  readily  changed  should  new 
information  become  available;  it  is  much  more  difficult  to  change  model 
code. 

It  is  important  to  keep  in  mind,  however,  that  although  expert 
systems  differ  from  traditional  programming  and  MOS  techniques,  they  should 
be  used  in  concert  with  these  techniques  and  be  viewed  as  an  a_ id  to  a 
forecaster. 

The  military  has  been  using  expert  systems  since  the  mid-1970's. 

These  systems  have  included  packages  such  as  the  Automatic  Target  Recognizer 
(4)  that  classifies  military  targets  from  sensor  images  and  rules.  Air 
Identification  (AIRID)  that  uses  various  aircraft  structural  rules  to  determine 
aircraft  type  (5),  and  the  Capability  Assessment  Expert  System  (CASES),  being 
developed  by  the  Space  and  Naval  Warfare  Command,  that  provides  decision 
support  to  the  U.S.  Pacific  fleet. 

The  above  military  applications  and  other  military  and  civilian 
expert  system  applications,  including  environmental  applications,  have 
several  common  starting  criteria  (6)  in  terms  of: 

•  Initial  requirements  for  the  expert  system 

•  Justification  for  the  expert  system 

•  General  characteristics  of  the  proposed  expert  system. 

We  have  listed  these  criteria  in  TABLE  2  to  illustrate  that  the 
development  of  a  low-visibility  expert  system  is  possible. 


TABLE  2.  General  Expert  System  Development  Criteria 
Versus  Low-Visibility  Expert  System  Effort. 


Generalized  Expert  System 
Development  Criteria 

Visibility  Effort 

Does  effort  require  only  cognitive 
skills? 

Yes: 

No  physical  (e.g.,  robotic) 
manipulations  were  undertaken. 

Does  a  base  of  genuine  expertise 
exist? 

Yes: 

Sufficient  numbers  of  meteo¬ 
rologists  (AWS  and  others)  with 
visibility  experience  exist. 

Significant  amounts  of  meteo¬ 
rological  literature  also  exist. 

Is  the  effort  simplistic  enough 
in  the  temporal  sense? 

Yes: 

Meteorologists  take  only  a  few 
hours  at  most  (or  usually  several 
minutes),  but  not  days  to  arrive  at 
a  low-visibility  decision.  However, 
how  to  deal  with  time  within  the 

KBES  will  be  difficult. 

Does  problem  area  appear  to  be 
well  structured? 

Qualified  yes:  Certainly  basic  research 

Is  still  occurring  In  the  visibility 
area;  however,  enough  knowledge 
exists  in  the  literature  and  within 

AWS  to  structure  a  knowledge  base. 

Will  the  expert  system  have  a 
high  payoff? 

Yes: 

An  expandable  KBES  concentrating  in 
low  visibility  first  and  then  moving 
to  other  areas  could  save  considerable 
amounts  of  time  and  effort  In  the 
Interpretation  of  visibility-related 
data  and  better  focus  base  weather 
resources  to  potentially  more  serious 
weather  problems. 

Is  human  expertise  scarce  or 
will  it  be  scarce  In  the  future? 

Yes: 

Meteorological  visibility  expertise 
currently  exists  within  AWS;  however, 
many  senior  civilian  AWS  meteorologists 
are  approaching  retirement  and  this 
special  expertise  may  be  lost.  In 
addition,  as  In  all  service  functions, 
AWS  personnel  are  shifted,  thus 
causing  a  scarcity  of  base-specific 
knowledge  that  only  longevity  at  a 
base  can  provide. 

Could  the  problem  area  use 
heuristic  solutions? 

Yes: 

A  heuristic  solution  refers  to  a 
solution  based  on  rules  of  thumb. 
Visibility  forecasting  and  other 
areas  of  meteorology  are  full  of 
rules  of  thumb  from  the  days  of 

J.J.  George  and  LaRoche  In  the  1950's 
to  the  present  Terminal  Forecaster's 
Notebook . 

Is  the  problem  of  manageable  size? 

Yes: 

To  include  precipitation  Into  the 

KBES  would  make  the  system  too  large 
for  a  proof -of -concept.  Thus,  we 
included  only  low  visibility  caused 
by  obstructions  to  vision. 

TABLE  2  attempts  to  answer  the  question  “Will  an  expert  system 
approach  work  in  a  low-visibility  (or  meteorological)  application?”  The 
table  indicates  that  low-visibility  (meteorology)  as  a  problem  area  is  ripe 
for  an  expert  system  application;  however,  certain  aspects  such  as  the  scope 
of  the  initial  system  and  the  user  interaction  with  the  expert  system  must  be 
kept  in  perspective. 

Perhaps  the  most  crucial  expert  system  development  question  centers 
around  the  scarcity  of  human  domain  (domain  in  this  case  referring  to  visibility 
-meteorology)  expertise.  Technology  transfer  and  the  codification  of 
knowledge  from  places  such  as  Air  Force  Global  Weather  Central,  AFGL,  and  the 
AWS-base  personnel  seems  to  be  a  prevalent  problem  not  only  in  the  Air  Force, 
but  in  some  of  the  other  environmental  areas. 

For  example,  GEOMET's  efforts  to  develop  a  coirmand,  control,  and 
communication-environmental  tactical  expert  system  in  modular  format  for  the 
Army  is  driven  by  the  need  to  rapidly  transfer  laboratory,  or  on  another 
level,  command  center  environmental  knowledge  down  to  the  small  field  combat 
units.  Other  GEOMET  environmental  efforts  for  the  Navy  have  clearly  indicated 
the  need  for  integrating  expert  system  usage  for  not  only  real-time  needs, 
but  as  a  training  vehicle  to  distribute  expert  knowledge  to  individual 
users. 

The  next  section  discusses  a  development  concept,  which  is  based  on 
the  coupling  of  the  human  and  machine  with  an  interface  that  complements  each 
side.  The  successful  development  and  implementation  of  the  KBES  will  require 
a  system  design  approach  that  is  based  on  graceful  growth  and  gradual 
enhancements. 

1.3  STEPS  IN  CREATING  AN  EXPERT  SYSTEM 

The  evolution  of  an  expert  system  proceeds  through  a  series  of 
steps  or  phases  designed  to  incrementally  improve  the  use  of  the  system 
knowledge.  The  five  phases  of  any  KBES  development  are  illustrated  in 
Fig.  3. 

It  is  important  to  note  that  the  design  of  an  expert  system  is  an 
ongoing  dynamic  process.  Even  though  on  paper,  generic  Phase  II  (for  example) 
may  be  complete  and  work  is  proceeding  on  other  phases,  the  creation  of  the 
expert  system  requires  knowledge.  Thus,  any  additional  knowledge  sources 
found  later  would  most  certainly  be  included  in  the  knowledge  base. 

A  discussion  of  each  generic  phase  follows. 


Generic  Phase  I:  Identification  of  Domain  Characteristics 


The  first  generic  step  in  developing  an  AI/KBES  is  the  character¬ 
ization  of  the  domain  (in  this  case  meteorology)  knowledge.  The  knowledge  can 
either  be  embedded  in  the  literature  or  acquired  from  experts.  Various 
techniques  can  be  used  to  acquire  knowledge  from  experts  including  observing 
the  expert  at  work  or  interviewing  the  expert. 


Generic  Phase  II:  Conceptualization  of  the  Expert  System  Architecture 

This  phase  involves  selection  of  an  architecture  appropriate  for 
system  design.  Two  general  approaches  include  frames  and  rules.  A  frame 
contains  knowledge  about  a  topic  in  the  form  of  slots;  rules  contain  individual 
if-then  or  similar  structures.  This  phase  also  involves  selection  of  either 
an  AI  language  such  as  LISP  or  PROLOG  or  a  shell,  which  is  a  software  package 
that  aids  in  expert  system  development  through  input  of  rules  and  knowledge 
much  like  Lotus  1-2-3  aids  in  graph  creation  through  data  input  by  the  user. 

Generic  Phase  III:  Placement  of  Knowledge  into  the  System 

In  this  phase,  the  various  logic  structure  goals/subgoals,  frames, 
etc.,  are  placed  into  the  expert  system.  Placement  of  rules  into  the  system 
early  enough  in  the  project  allows  for  any  pitfalls  to  be  uncovered.  Generally 
early  placement  of  rules  into  the  expert  system  also  allows  for  limited 
evaluation  of  the  various  rule  or  logic  paths. 


Generic  Phase  IV:  Expert  System  Evaluation 


A  formal  zed  system  evaluation  can  help  both  the  user  and  the 
developer  in  terms  of  interaction  and  refinements.  Typically,  evaluations 
are  quantitative  in  nature,  although  several  KBES  evaluations  are  beginning 
to  become  more  qualitative. 


KBES  evaluations,  however,  have  shied  away  from  the  field  evaluation 
concept  that  is  used  by  many  scientific  organizations.  Instead,  many  KBESs 
remain  buried  in  a  laboratory  environment  and  never  get  a  full  field  evaluation 


Generic  Phase  V:  Training 

Typical  KBESs  involve  some  level  of  user  interaction  that  requires 
some  degree  of  training.  This  training  could  involve  merely  a  system  or 
module  overview  or  a  hands-on  user  interaction  with  the  system  or  module,  all 
under  control  of  a  team  member. 

I  Low-Visibility  KBES  Development 


!  Development  of  the  low-visibility  KBES  (hereafter  called  "Zeus") 

[  follows  a  similar  progression  of  the  phases  just  described.  Fig.  4  shows  the 

(  various  steps  in  creating  Zeus  and  key  issues  under  each  step.  To  appreciate 


truly  makes  up  an  expert  system  design,  we  follow  and  discuss  each  step  in 
the  following  sections.  The  reader  will  notice  some  slight  differences  in  the 
generic  approach  that  can  be  applied  to  any  system  versus  Zeus,  but  on  the  whole, 
the  approaches  are  the  same. 
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Section  2.0 

IDENTIFICATION  OF  DOMAIN  CHARACTERISTICS 


In  this  section,  we  describe  the  domain  that  characterizes  the  param 
eters  used  in  low-visibility  forecasting  and  the  operating  environment  to 
which  the  expert  system  will  be  constrained.  The  discussion  includes  the 
fol lowing  topics : 

•  The  meteorological  phenomena  of  fog 

•  Base-specific  meteorology 

•  Base  weather  forecasting  time  constraints 

•  Available  computer  resources. 

2.1  IDENTIFICATION  OF  THE  METEOROLOGICAL  PR0BLEM--F0G 

2.1.1  Background 

Visibility  is  of  considerable  importance  to  the  Air  Force  and 
military  community  in  general,  mainly  because  of  the  transportation-oriented 
operations  that  may  be  hindered  or  stopped  altogether  by  visibility  below 
certain  limits.  Visibility  throughout  history  has  played  a  significant  role 
in  many  areas  including  battles  and  transportation  disasters.  For  example, 
the  1986  aircraft  collision  at  Tampa's  airport  was  directly  influenced  by  low 
airport  visibility  at  the  time  of  the  accident. 

Obstructions  to  vision  have  been  defined  as  nonprecipitating 
phenomena  that  reduce  visibility.  Examples  include  haze,  smoke,  blowing  dust 
and  sand,  blowing  and  drifting  snow,  and  fog. 

Haze  has  been  considered  as  a  form  of  atmospheric  pollution  and  is 
composed  principally  of  very  small  salt  crystals  and  dust  particles.  Aqueous 
haze  droplets  tend  to  form  on  hydroscopic  condensation  nuclei.  Haze  droplets 
develop  as  the  relative  humidity  increases  above  a  certain  critical  point, 
which  is  50  percent  (7).  Smoke  is  a  result  of  industrial  processes,  coal  and 
wood  burning,  and  forest  fires.  Dust  is  composed  of  thousands  of  small  soil 
or  sand  fragments  that  are  carried  to  great  heights  by  thermals  and  to  great 
distances  by  winds.  Blowing  sand  is  regarded  as  larger  soil  or  sand  particles 
which  are  carried  or  supported  by  surface  winds.  Blowing  snow  is  generally 
character i zed  by  a  drier  snow,  which  is  carried  aloft  by  gusty  surface  winds. 
Both  horizontal  and  vertical  visibility  are  restricted.  Drifting  snow,  to  a 
lesser  extent,  affects  only  the  horizontal  visibility. 


During  the  very  early  stages  of  Zeus  development,  we  decided  to 
limit  the  system  to  obstructions  to  vision  (specifically,  fog  and  haze). 
Precipitation- induced  low  visibility  was  not  considered  because  this  would 
require  a  very  large  knowledge  base  and  possibly  the  introduction  of  pattern¬ 
matching  techniques.  We  believe  that  the  large  knowledge  base  at  this  time 
would  defeat  the  purpose  of  this  effort,  which  was  to  establish  a  proof-of- 
concept  in  meteorological  AI;  thus,  we  chose  the  limited  fog  and  haze  phenomenon. 

Visibility  restrictions  caused  specifically  by  fog  and  haze  are  of 
major  concern  to  Air  Force  operations.  Numerous  studies  provide  descriptions 
on  how  visibility  restrictions  affect  operations  such  as  takeoffs  and  landings 
as  well  as  battlefield  and  aerial  refueling. 

2.1.2  Advection  Fog  Formation  Conceptual  Model 

Fogs  have  been  classified  into  various  types  by  early  authors  such 
as  Willett  (9)  and  George  (10)  and  by  recent  papers  such  as  Welch  et  al. 

(11).  Fifteen  categories  of  fog  exist;  however,  the  two  broad  types  of  fog 
are  advection  and  radiation.  True  advection  fogs  are  sea  fog,  arctic  steam 
mists,  and  snow-surface- Induced  fogs  (i.e.,  fogs  caused  by  air  with  a  dewpoint 
above  freezing  traveling  over  a  snow  surface).  Actually  these  true  types  of 
advection  fog  do  not  occur  over  the  United  States  very  often. 

Alternatively,  the  literature  has  provided  a  looser  definition  of 
advection  fog  to  represent  fog  caused  by  sufficient  moisture  to  ensure 
saturation  after  a  reasonable  amount  of  cooling.  Typically,  parcels  of  air 
are  moved  over  surfaces  colder  than  themselves  and  therefore,  cool,  producing 
a  fog.  Thus,  fogs  that  depend  on  wind  to  alter  temperature  or  moisture 
content  (or  both)  of  the  air  parcels  in  such  a  manner  that  saturation  is 
achieved  are  called  advection  fogs.  The  simplest  definition  contained  in  the 
literature  is  the  one  we  have  adopted  to  avoid  confusion;  The  air  in  which 
the  fog  forms  must  have  had  at  least  a  short  path  over  water  since  the 
preceding  day.  A  search  of  the  literature  indicates  that  many  authors  refer 
to  this  as  advection- radiation  fog;  however,  we  will  call  it  advection 
fog  for  simplicity's  sake. 

Source  regions  for  advection  fog  naturally  include  areas  within 
approximately  200  mi  of  a  large  body  of  water.  Occasionally,  however, 
advection  fogs  have  been  observed  "being  fed"  by  ground  moisture  coming  from 
an  area  that  has  experienced  thunderstorms  or  rainshowers  (12). 

Advection  fog  is  formed  under  synoptic  conditions  that  allow  for 
the  boundary- layer  transport  of  moist  air  into  an  area  with  relatively 
cloud-free  skies  (hence,  the  "radiation  influence").  Therefore,  a  key 
parameter  in  forecasting  advection  fog  focuses  on  the  use  of  the  surface  wind 
and  boundary- layer  trajectories.  If  the  surface  and  boundary- layer  flow  is 
off  a  water  body,  then  depending  on  the  synoptic  situation,  there  is  an 
Increasing  probability  of  advection  fog. 
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By  having  an  understanding  of  the  influence  of  surface  and  boundary- 
layer  windflow,  one  can  then  introduce  other  forecast  paraneters  into  the 
analysis.  Even  though  winds  must  be  sufficient  to  transport  moisture  into  an 
area,  the  winds  cannot  be  so  strong  that  they  tend  to  warm  the  boundary  layer 
and  reduce  condensation,  thereby  reducing  the  fog  threat. 

Physical  evidence  of  the  dissipation  of  advection  fog  due  to  wind 
has  been  presented  by  Jiusto  (13),  who  links  changes  in  fog  density  to  the 
Richardson  number  that  is  defined  as 

m  .  l  Wdz  in 

9  (du/dz)2 

where 


d0/dz  represents  the  vertical  potential  temperature  (0,  in  K)  gradient 
du/d z  represents  the  windspeed  (u,  in  m)  change  with  height 
g  represents  gravity. 

Typically,  low  visibilities  have  tended  to  occur  when  R^  >  0.5.  With  R^  <  0.5, 
vertical  mixing  is  dominant;  a  strong  bound ary- 1 ayer  wind  will  keep  the 
bound  ary- layer  well  mixed  and  dd/dz  low.  Both  the  cooling  and  the  moisture 
from  the  surface  are  distributed  upward  and  this  produces  stratus  clouds 
rather  than  fog.  This  suggests  that  critical  bound ary- layer  and  surface 
windspeeds  must  be  established  within  Zeus  to  reflect  this  physical  reasoning. 

Another  parameter  of  importance  involves  the  amount  of  cloud  cover 
between  a  forecast  site  and  the  moisture  source.  Large  amounts  of  intervening 
cloud  cover  will  tend  to  limit  any  cooling  of  the  boundary  layer,  thereby 
restricting  fog  formation.  Fig.  5  dramatically  illustrates  the  effect  of 
variable  cloudiness  during  an  advection  fog  case  in  the  New  York  City  area. 

A  cloud  bank  passed  near  and  overhead  of  the  observing  site  from  around  2200  EST 
to  2345  EST.  Visibility  clearly  improved  with  the  arrival  of  the  cloud 
bank  and  was,  in  fact,  directly  correlated  with  the  level  and  thickness  of 
the  cloud  bank.  The  cloud  bank  effectively  reduced  outgoing  radiation  from 
about  3.5  mW  cm-2  to  0.5  mW  cm-2. 

Advection  fog  is  also  characterized  by  well-defined  boundaries, 
which  are  usually  (assuming  the  absence  of  thin,  higher  level  clouds)  evident 
in  both  the  visible  and  IR  satellite  imagery.  A  windward  edge  or  side  of  an 
advection  fog  area  remains  quasi-stationary,  but  the  edges  and  lee  side 
typically  extend  various  distances  outward.  A  shift  of  the  wind  or  the 
introduction  of  drier  air  may  alter  the  entire  fog  pattern. 

With  the  above  physical  factors  in  mind,  we  developed  a  conceptual 
model  of  a  forecaster's  probable  thinking  when  deciding  on  an  advection  fog 
forecast.  This  model,  shown  in  Fig.  6,  was  developed  prior  to  the  formal 
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interviews  in  the  knowledge  acquisition  phase  and  served  as  a  framework  for 
discussions  with  AWS  personnel.  The  model  indicates  a  chain-of-thought  based 
on  physical  reasoning,  with  the  key  parameter  being  the  advection  of  moisture 
from  a  moisture  source. 

Other  specific  parameters  included  the  windspeed  itself,  which  can 
be  considered  in  the  broad  sense  a  measure  of  bound ary- 1 ayer  turbulence,  and 
the  clouds.  Other  rel ated  factors,  which  at  this  stage  referred  to  any 
peculiar  base-specific  meteorology,  are  also  considered. 

We  were  immediately  faced  with  several  problems,  including: 

1.  How  do  we  identify  the  synoptic  situations  favorable 
for  advection  fog? 

2.  How  do  we  "kickout"  of  the  conceptual  model  if  necessary 
conditions  are  not  present? 

These  problems  were  carefully  noted  for  special  attention  in  later  phases. 

A  similar  conceptual  model  was  developed  for  radiation  fog  and  is 
discussed  in  the  next  subsection. 

2.1.3  Radiation  Fog  Formation  Conceptual  Model 

Radiation  fog  occurs  in  an  air  mass  when  sufficient  cooling  occurs 
due  to  radiative  loss  of  sensible  heat.  Considerably  more  literature 
exists  on  radiation  fog  than  on  advection  fog.  A  search  of  this  literature 
indicates  several  key  parameters  for  the  formation  of  radiation  fog: 

•  Clear  or  mostly  clear  skies 

•  Adequate  relative  humidity  in  the  surface  layer 
(lowest  100  m)  [wet  ground  may  be  substituted] 

•  Lack  of  strong  surface  and  boundary-1 ayer  winds. 

It  appears  from  the  literature  that  radiation  fog  formation  is  the 
end  product  of  a  very  complex  set  of  processes  within  the  surface  and  boundary 
1 ayer . 


Assuming  relatively  clear  skies,  the  net  cooling  of  the  ground 
surface  begins  just  prior  to  sunset.  The  radiational  balance  at  that  time 
appears  as  follows: 

^  ^Sun  +  ^Sky  +  ^o  +  Ri  (2) 
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where: 


R[  =  heat  lost  by  long-wave  radiation  to  sky 

^Sun  =  short-wave  radiation  from  sun 

RSky  =  short-wave  radiation  from  sky 

H0  =  heat  conducted  from  upper  ground  levels 

R|_  =  long-wave  radiation  received  from  atmosphere. 

Net  cooling  includes  cooling  of  the  air  at  and  near  the  ground.  When  the 
ground  has  cooled  to  the  dewpoint  temperature,  dew  deposition  begins. 
Sufficient,  but  not  overwhelming  turbulence  is  needed  in  the  surface  layer  to 
bring  fresh  supplies  of  air  to  the  ground.  Monteith  (14),  for  example, 
suggests  that  windspeeds  less  than  0.5  m  sec'l  are  insufficient  to  promote 
dew  deposition. 

If  sufficient  turbulence  is  present,  the  amount  of  water  vapor 
decreases  (falling  dewpoints),  so  the  temperature  must  continue  to  fall 
before  fog  forms.  This  is  a  critical  time  of  delicate  balance  in  the  fog 
formation  cycle.  Usually,  temperatures  and  dewpoints  continue  to  fall  and 
fog  forms,  first  in  very  thin  layers  separated  by  clear  air  from  the  ground. 
Radiational  cooling  from  the  fog  layers  themselves  leads  to  thickening  and 
vertical  extensions.  Radiational  cooling  at  the  fog  top,  which  could  range 
from  10  to  300  m,  may  be  as  much  as  2  *C  in  30  min  (15). 

Our  literature  review  also  indicated  many  overnight  temperature 
prediction  schemes  that  incorporate  dry-bulb  and  dewpoint  temperatures  near 
sunset  and  extrapolate  these  values  through  to  sunrise  to  determine  a  critical 
fog  temperature.  The  technique  of  Craddock-Pri tchard  (16)  for  example,  uses 
the  following  linear  regression  equation: 

Tp0g  =  0.044Ti2  +  0.844Tdi2  -  0.55  +  A 
TFog  =  T  +  A 

where: 

J\2  =  temperature  at  12  Z 
Tdl2  =  dewpoint  temperature  at  12  Z 

Y  =  calculated  value  of  expression  (3)  in  nomogram  form 
A  =  lookup  factor  (in  degrees)  based  on  sky  cover  and  mean 
geostrophic  forecasted  windspeeds  for  18Z,  00Z,  and  06Z. 

The  next  step  involves  taking  Tp0g  and  using  it  such  that 

E  =  ^Fog  ”  Tmin  (5) 

where 

Tmln  relates  to  another  Craddock-Pri tchard  lookup  table  and 

E  =  final  value  that  is  then  compared  to  an  E  table  fog  risk  (high, 
moderate,  or  low) . 


(3) 

(4) 


Other  graphical  methods  exist  such  as  George's  method  (17),  which 
has  been  used  by  several  AWS  personnel . 

George's  empirical  equation  is: 

Tpog  =  6.13  loge  0.18  (S  +  D)  +  3.2  (6) 

where : 

Tpog  =  time  of  fog  formation  in  hours  after  sunset  when  visibility 
is  less  than  or  equal  to  1  mi 
S  *  number  of  sunshine  hours 
D  =  dewpoint  depression  at  sunset. 

Conditions  for  use  of  this  formulation  include: 

•  Reasonable  amounts  of  cloudiness  during  day 

•  Gradient  wind  below  25  mi/h 

§  Decreasing  cloudiness  at  night. 

This  formulation  is  based  on  several  years  of  meteorological  data  from 
Atlanta,  Georgia,  and  has  been  confirmed  at  other  cities  such  as  Nashville, 
Louisville,  and  Chicago. 

As  with  any  statistical  methods,  the  results  (the  regression 
equations)  express  only  mean  relationships.  Thus,  caution  should  be  exercised 
in  using  these  equations  as  a  predictor  alone.  However,  modifications  to 
these  equations  in  terms  of  meteorological  parameters  that  may  influence 
dewpoint  depression  (for  example)  would  be  a  very  powerful  use  of  statistics 
and  knowledge. 

Any  conceptual  model  of  radiation  fog  forecasting  must  consider  the 
predominance  of  statistical  methods  of  forecasting  radiation  formation. 

Fig.  7  shows  our  conceptual  radiation  fog  model  based  on  the  physical 
reasoning  presented  above. 

This  conceptual  model  was  developed  to  guide  our  interview  process 
and  includes  such  general  physical  characteristics  as  the  clear  skies  and 
moist  ground  features  discussed  earlier. 

As  with  the  advection  fog  conceptual  model,  we  were  faced  with 
several  problems  (as  listed  in  the  figure),  namely: 

1.  How  do  we  deal  with  surface  layer  moisture? 

2.  How  do  we  identify  the  favorable  synoptic  situations 
for  radiation  fog? 
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Figure  7. 


Conceptual  model  (based  on  physical  reasoning) 
of  a  radiation  fog  forecast. 
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The  first  problem  was  directly  addressable  by  our  domain  experts. 

The  second  problem  is  common  to  both  advection  and  radiation  fog  and  is  the 
topic  of  a  separate  subsection  below. 

A  third  concern  that  arose  at  this  stage  was  how  to  deal  with  the 
multiplicity  of  "kickouts"  that  the  system  would  require  should  a  condition 
not  be  sufficient.  Thus,  a  preliminary  decision  was  reached  based  on  these 
two  conceptual  models  to  analyze  the  possibility  of  advection  fog  first, 
simply  because  of  the  moisture,  and  boundary  layer  assessments  that  could  be 
made  early  in  the  program  run  and  the  commonality  of  such  assessments  in 
radiation  fog.  This  led  to  presenting  the  conceptual  models  to  AWS  personnel 
in  the  advection  and  then  radiation  fog  order,  with  synoptics  being  intertwined 
in  the  discussion. 

2.1.4  Synoptic  Conceptual  Model 

During  the  development  of  the  conceptual  models  for  radiation  and 
advection  fog,  it  was  noted  that  a  recurring  theme  of  the  synoptic  situation 
driving  the  local  meteorology  occurred;  thus,  special  thought  was  given  to 
developing  a  general  conceptual  synoptic  model.  The  danger  in  developing 
such  a  model,  however,  was  to  "go  overboard"  and  create  the  "catch-all" 
synoptic  expert  system--which  is  really  beyond  the  scope  of  this  proof-of- 
concept  effort. 
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A  more  detailed  description  of  base-specific  meteorology  appears  in 
a  later  section;  however,  several  key  synoptic  features  (identified  in  the 
literature)  are  prerequisites  for  fog  formation: 

•  High/ridge  in  a  favorable  position  to  permit  at  least 
weak  oceanic  flow  (advection) 

•  High/ridge  in  a  favorable  position  overhead,  or  nearby 
to  allow  for  subsidence,  and  clear  skies  (radiation) 
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•  Lows  not  expected  to  affect  weather  next  12  h 

•  Fronts  (except  backdoor  cold  front)  not  in  a  position 
to  threaten  area 

•  Backdoor  cold  front  (front  moving  southward  from  the  north 
or  northeast  of  a  base)  in  a  position  to  threaten  area. 

At  this  early  stage,  due  to  proof-of-concept  limitations,  we 
introduced  the  idea  of  the  AWS  forecaster  (the  user)  deciding  where  the 
synoptic  features  will  go.  We  were  forced  to  "draw  this  line"  because  the 
number  of  rules  on  movement  and  strength  of  synoptic  systems  alone  were  very 
complex  and  could  involve  an  entirely  separate  expert  system. 

Fig.  8  shows  the  synoptic  conceptual  model .  One  interesting 
aspect  that  the  development  of  this  conceptual  model  indicated  is  the  need 
for  a  temporal  sense  of  influencing  synoptic  features.  In  other  words,  the 
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timing  of  hiqhs  movinq  into  New  England  influences  the  occurrence  of  advection 
fog  and  should  be  included  in  any  mid-Atlantic  fog  expert  system. 


In  addition,  this  conceptual  model  forced  us  to  begin  to  consider 
how  the  expert  system  would  actually  position  the  various  synoptic  systems 
(thus,  the  concept  of  "regions"  that  is  introduced  in  Section  3.0). 

The  conceptual  model  also  forced  us  to  realize  that  synoptic 
systems  can  influence  base  weather  in  three  common  sense  ways: 

•  Cause  nc  change 

•  Cause  a  change  for  the  better 

•  Cause  a  change  for  the  worse. 

For  example,  a  cold  frontal  passage  (continuous  movement--no  stalling) 
should  cause  a  change  for  the  better  because  these  fronts  tend  to  clear  an 
area  of  existing  weather.  On  the  other  hand,  backdoor  cold  fronts  (which  slide 
down  the  East  Coast  from  the  north  or  northeast)  usually  cause  a  change  for 
the  worse  because  their  passage  allows  for  the  advent  of  a  moist  northeasterly 
(oceanic)  flow  conducive  to  advection  fog  formation. 


The  major  problem  with  the  synoptic  conceptual  model  is  how  to  link 
or  use  the  information  in  the  other  conceptual  models.  This  is  discussed  in 
the  knowledge  representation  section  in  terms  of  a  hierarchy  of  operations. 


As  an  indirect  result  of  developing  the  conceptual  synoptic  model, 
we  also  began  to  realize  that  the  fog  problem  needs  to  be  carried  through  a 
full  cycle.  We  began  to  examine  how  synoptic  meteorology  could  affect  the 
dissipation  of  fog  and  thus,  developed  the  fog  dissipation  conceptual  model. 


One  interesting  aspect  of  the  fog  forecasting  problem  that  was 
initially  neglected  during  the  identification  phase  was  the  forecasting  of 
fog  dissipation. 

Our  literature  search  found  several  methods  of  predicting  fog 
dissipation.  All  methods  were  based  on  daytime  fog  being  dispersed  by 
incoming  solar  radiation.  At  night,  fog  is  typically  decreased  by  outgoing 
radiation  being  cutoff  by  intervening  clouds.  Fog  may  be  cleared  anytime  by 
either  increasing  the  wind  or  advecting  in  drier  air,  both  of  which  are 
characteristics  of  typical  (nonstalling)  cold  fronts  on  the  East  Coast. 

It  appears  from  climatology  that  most  radiation  fogs  in  the  coastal 
plain  south  of  New  Jersey  "break"  (visibility  increases  to  greater  than  1  mi) 
within  1  to  4  h  after  sunrise  (18).  Typically,  the  earlier  the  fog  forms 
(and  barring  any  changes),  the  denser  it  will  be.  East  Coast  nomograms  of 
fog  dissipation  times  based  on  density  have  been  developed  (18). 

The  conceptual  dissipation  model  appears  in  Fig.  9. 
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The  four  conditions  listed  can  either  independently  influence  the 
dissipation  of  the  fog  or  jointly  affect  dissipation.  Key  questions  to  be 
answered  include  how  quickly  will  the  fog  dissipate  and  to  what  degree  will 
the  dissipation  occur  (improve  to  1,  2,  or  3+  mi)?  Also,  our  conceptual 
model  indicates  an  "automatic"  improvement  feature  that  allows  for  clearing 
after  the  passage  of  a  cold  front.  This  was  the  result  of  developing  the 
change,  no-change  concept  in  the  synoptic  model. 

We  next  examined  the  meteorology  for  each  selected  USAF  base. 

2.2  IDENTIFICATION  OF  BASE-SPECIFIC  METEOROLOGY 


Three  bases  (shown  in  Fig.  10)  were  selected  for  the  expert  system 
proof-of-concept  evaluation.  These  bases  and  their  accompanying  weather  code 
and  USAF  flight  functions  are: 

•  Oover  Air  Force  Base  (AFB) ,  Delaware,  DOV,  MAC  (C-5,  C-130) 

t  Seymour  Johnson  AFB,  North  Carolina,  GSB,  TAC  (F-4, 

KC-135 ,  KC-10) 

•  Simmons  Army  Airfield  (Fort  Bragg),  North  Carolina, 

FBG  (hel icopters) . 

Various  visibility  criteria  were  obtained  from  the  Terminal  Forecast 
Reference  Notebook  (TFRN)  for  each  base  and  are  listed  in  TABLE  3.  These 
criteria  were  subsequently  modified  slightly  to  reflect  the  more  general 
categories  that  appear  in  TABLE  4. 

A  brief  description  of  base-specific  meteorology  appears  below. 

2.2.1  Dover  AFB  (DOV) 


Dover  AFB  is  located  approximately  4  mi  southeast  of  Dover  and  is 
28.6  ft  above  sea  level.  The  Delaware  Bay  is  3  mi  to  the  east  and  the 
Chesapeake  Bay  lies  35  mi  to  the  west  (see  Fig.  11).  The  Atlantic  Ocean  at 
the  mouth  of  the  Delaware  Bay  is  25  mi  southeast.  With  the  proximity  of 
water  nearby,  there  is  great  concern  over  advection  fog. 

Beginning  with  the  microscale,  the  base  sits  on  the  southern  end  of 
a  slight  ridge.  Slight  cold  air  drainage  is  experienced  on  cloud-free  nights 
with  light  winds.  According  to  the  TFRN,  this  effect  occasionally  keeps  the 
marshes  around  the  base  covered  with  radiational  fog  while  the  base  itself  is 
free  of  fog. 

The  TFRN  also  mentions  the  following  factors  pertaining  to  fog 
conditions: 


Winds  from  the  southwest  through  north  rarely  produce  fog 
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TABLE  3. 


Visibility  Criteria  for  the  Three  Selected  Airbases 
(Source:  Base  Terminal  Forecast  Handbooks  or 
Weather  Support  Plans). 


Cei 1 ing/Visibi 1 ity 


Result/Impl ication 


Seymour  Johnson  Air  Force  Base  (GSB)  Criteria 


3000/5 

3000/3 

1500/3 


700/2 


500/1.5 


100/0.25 


Flight  control  checks  limited 

VFR  limitation 

Category  E  pilot  mission  minimums 
(defined  as  a  pilot  with  all 
initial  and  sequential  training 
prior  to  formal  instrument 
evaluations) 

Category  D  pilot  mission  minimuns 
(defined  as  a  pilot  having 
completed  instrument  evaluation) 

Category  C  pilot  mission  minimums 
(defined  as  a  pilot  having 
50  actual  flying  hours,  and 
500  total  hours) 

Absolute  airfield  minimum 


KC-10  tanker  minimums  are  not  yet  available. 

Fort  Bragg  (Simmons)  Army  Air  Field  (FBG)  Criteria 
300/3/4  Base  minimums 

Dover  Air  Force  Base  (DOV)  Criteria 

1000/2  Training  mission  minimums 

Landing  minimums 


200/1/2 


TABLE  4.  Modified  Airbase  Visibility  Criteria  for  this  Study. 


GSB  and  FBG 


Visibility 

>  3 

>  1  and  <  3 
<  1 


Zeus  Category 
3 
2 
1 


Summertime  sea  breezes  add  to  the  moisture  concerns 
in  the  boundary  layer  and  hence,  to  the  possibility 
of  fog. 


In  general  terms,  spring  is  a  transition  time  with  several  backdoor 
cold  frontal  passages  resulting  in  periods  of  dense  fog.  During  early 
spring,  land-sea  temperatures  are  basically  equivalent,  but  the  conflict 
between  the  retreating  Labrador  nearshore  oceanic  current  and  the  Gulf  Stream 
produces  advection  fog.  Radiation  fog,  which  is  moved  into  the  area  from 
offshore  waters,  is  less  common. 

The  Bermuda  High  causes  radiation  fog  and  haze  conditions  in  the 
summer  with  average  dewpoints  in  the  60s.  Visibilities  typically  drop  to  2 
to  3  mi  in  fog  and  haze  overnight,  except  in  early  September  where  sea 
surface  temperatures  become,  on  an  average,  warmer  than  the  daily  minimum. 
Therefore,  southeasterly  winds  at  Dover  would,  in  late  summer  and  early  fall, 
transport  warm  moist  air  over  cooler  land,  thus  causing  fog. 

This  fall-type  condition  persists  through  winter  and  into  early 
spring  with  the  daily  maximum  temperature  typically  falling  well  below  the 
sea  temperature.  This,  in  turn,  causes  widespread  fog  (and  stratus)  that  may 
be  hard  to  break. 

Statistics  from  the  AWS  Climatic  Station  Brief  indicate  that 
visibility  due  to  fog  alone  (fog  alone  carried  on  the  hourly  or  special 
observation)  occurs  annually  at  a  mean  rate  of  183  days  in  an  average  year. 
TABLE  5  gives  a  breakdown  by  month  of  the  mean  number  of  fog  days  per  month; 
TABLE  6,  also  taken  from  the  AWS  Climatic  Briefs,  gives  a  further  breakdown 
of  ceiling  and  visibility  category  by  percentage  and  cross  referenced 
by  month  (and  time).  From  both  tables,  it  appears  that  the  highest  incidences 
of  fog  (climatologically)  appear  in  sunnier  and  early  fall,  and  as  common 
meterological  sense  would  dictate,  occur  with  maximum  frequency  between 
6  a.m.  and  8  a.m. 

2.2.2  Seymour  Johnson  AFB  (GSB) 

Seymour  Johnson  AFB  is  located  in  the  western  section  of  the 
North  Carolina  coastal  plain  at  an  elevation  of  109  ft.  Terrain  varies  from 
50  to  200  ft  around  the  base.  The  base  is  located  approximately  125  mi  from 
the  ocean.  Regionally,  the  base  is  influenced  by  downslope  winds  from  the 
northwest.  Converserly  easterly  winds  tend  to  "dam"  the  air  up  against  the 
Appalachians  and  cause  low  ceilings  and  precipitation.  Fig.  12  from  the 
TFRN,  depicts  the  larger  regional  scale  terrain  features.  The  base  weather 
is  locally  influenced  by  the  Neuse  River,  which  is  located  near  the  west  end 
runway  08/26.  Fog  typically  forms  over  the  river  and  could  possibly  advect 
over  the  field  depenaing  on  the  wind  direction. 

The  TFRN,  along  with  other  base-specific  guidance  documents,  indicate 
several  key  parameters  for  determining  low  visibility.  They  are: 

•  Temperature,  dewpoint,  and  dewpoint  spread 
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TABLE  5.  Dover  AFB  Low  Visibility  Climatology. 


Month 


Mean  number  of  days  of  fog 


i  January 

13 

1  February 

12 

\  March 

14 

J  April 

13 

3  May 

16 

J  June 

16 

I  July 

18 

1  August 

20 

?  September 

18 

>  October 

16 

%  November 

14 

|  December 

13 

Note:  Based  on  32  years  of  data. 


TABLE  6 


Percent  Frequency  of  Low  Visibility/Ceiling 
by  Two  Hourly  Periods  (by  Month). 


Figure  12.  Regional  map  (profile  view)  of  Seymour  Johnson's  elevation  relationship  to  the  rest 
3  of  North  Carolina^ 


•  Sky  cover  (overnight) 

•  Wind  direction  and  speed 

0  Precipitation. 

A  search  of  a  more  detailed  climatological  record  of  fog  by  month 
(19)  revealed  some  interesting  aspects  of  Seymour-Johnson  fog. 

In  January  (based  on  8  year's  worth  of  data),  83  percent  of  the 
below  2  mi  occurences  were  associated  with  precipitation  the  previous  day  and 
it  was  noted  that  most  fogs  were  radiational  in  nature.  This  trend  continued 
throughout  the  winter  (81  percent  occurrence  in  February,  89  percent  in 
March)  with  dewpoint  spread  becoming  increasingly  important. 

In  the  spring,  fog  forecasts  are  influenced  by  advection  of  maritime 
air  down  the  coast  due  to  backdoor  cold  fronts.  Advection  fogs  appear  to 
dominate  during  this  time  of  the  year. 

In  the  summer,  light  winds  promote  radiation  fog  with  numerous 
cases  of  persistent  radiation  fog  noted.  Radiation  fog  lifting  to  haze  (2  to 
3  mi )  by  midday  is  also  a  common  problem.  The  Neuse  River  also  causes 
problems  in  late  summer-early  fall  in  terms  of  acting  as  a  generator  of  local 
fog  banks. 

Light  winds  continue  to  be  a  major  factor  contributing  to  fog  in 
the  fall  with  many  instances  of  cold  fronts  followed  by  stagnant  large  areas 
of  high  pressure  that  can  persist  over  the  area  for  days. 

Statistics  from  the  AWS  Climatic  Brief  indicate  that  visibility  due 
to  fog  alone  occurs  annually  at  a  mean  rate  of  202  days.  TABLE  7  provides  a 
breakdown,  by  month,  of  the  mean  number  of  fog  days;  TABLE  8  gives  a  further 
breakdown  of  ceiling  and  visibility  category  by  percentage  and  cross-referenced 
by  time  and  month.  Both  tables  indicate  that  the  dominate  fog  occurrences 
are  in  late  summer-early  fall  with  the  time  of  occurrence  between  6  a.m.  and 
8  a.m.  with  a  secondary  maximum  between  3  a.m.  and  5  a.m. 

2.2.3  Simmons  AAF  (Fort  Bragg)  (FBG) 

Sirmions  AAF  is  located  approximately  20  mi  from  Pope  AFB  in  the 
vicinity  of  Fayetteville,  North  Carolina.  The  elevation  is  242  ft. 

Simmons  is  located  geographically  within  the  transition  zone  between  the 
coastal  plains  and  Piedmont  Plateau  of  North  Carolina. 

The  Atlantic  Ocean  ranges  from  approximately  170  nmi  east-northeast 
of  the  base  to  approximately  80  nmi  southeast  of  the  base.  Several  ponds  and 
potential  cold  air  drainage  flow  areas  surround  the  base. 


TABLE  7.  Seymour  Johnson  AFB,  Low  Visibility  Climatology. 


Month 

Mean  number  of  days  of  fog 

January 

13 

February 

12 

March 

13 

April 

12 

May 

18 

June 

19 

July 

22 

August 

23 

September 

22 

October 

14 

November 

16 

December 

13 

Total 

202 

Note:  Based  on  22  yr  of  data. 


TABLE  8.  Percent  Frequency  of  Low  Visibility/Ceiling 
by  Two  Hourly  Periods  (by  Month). 
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During  the  latter  part  of  spring,  through  summer,  and  into  early 
fall,  radiation  fog  appears  to  dominate  over  advection  fog.  There  appears  to 
be  nothing  unusual  about  these  radiation  fogs  other  than  the  TFRN  noting  that 
radiation  fogs  are  typically  thin.  Summertime  haze  is  also  present  on  many 
days.  Summertime  fogs  also  form  after  rainshowers,  particularly  with  southwest 
flow. 

Later  in  the  fall  and  throughout  the  winter,  advection  fogs  begin 
to  dominate.  The  fogs  are  caused  by  a  combination  of  an  easterly  gradient 
transporting  warm  moist  air  inland  from  the  coast  and  the  associated  slight 
upslope  motion  (0  to  242  ft).  Easterly  gradients  are  typically  caused  by 
highs  located  in  the  New  England  area.  Thus,  care  is  needed  to  determine 
whether  a  true  easterly  gradient  exists,  as  occasionally  easterly  winds  in  a 
tight  gradient  may  not  have  an  Atlantic  trajectory.  Steam  fog  may  also  form 
over  many  of  the  nearby  drop  zones  (due  to  the  presence  of  the  ponds)  and 
never  affect  the  base  itself. 

Statistics  from  the  AWS  Climatic  Brief  Indicate  that  visibility  due 
to  fog  alone  occurs  annually  at  a  mean  rate  of  200  d  in  an  average  year. 

TABLE  9  provides  a  breakdown,  by  month,  of  the  mean  number  of  fog  days; 

TABLE  10  gives  a  further  breakdown  of  ceiling  and  visibility  category  by 
percentage,  cross-referencd  by  time  and  month.  An  examination  of  both 
tables  reveals  that  the  highest  incidence  of  fog  (climatologically)  appears 
in  midsummer  with  July  and  August  being  the  highest  months.  The  most 
critical  time  for  fog  appears  to  be  from  6  a.m.  to  8  a.m.,  although  It 
is  interesting  to  notice  in  TABLE  10  the  flattening  of  the  percent  data  in 
February  to  a  broad  peak  between  6  a.m.  and  11  a.m.,  thus  indicating  in 
a  general  sense  the  problem  of  advection  fog  dissipation. 

2.3  IDENTIFICATION  OF  BASE  WORK  CONCERNS 

Toward  the  end  of  the  identification  stage  we  began  to  list 
various  possible  problems  that  the  bases  could  have  in  forecasting  fog. 

These  problems  or  factors  were  not  directly  related  to  the  meteorology  itself 
(i.e.,  the  physics),  but  Instead  to  some  of  the  human  factors  that  go  into 
creating  a  forecast.  The  factors  can  therefore  be  called  human  system 
design  concerns.  The  factors  appear  In  TABLE  11. 

Perhaps  the  greatest  factor  that  we  could  get  a  sense  of  during 
this  identification  phase  was  the  hurry  factor.  (This  subsequently  was 
confirmed  over  and  over  during  the  later  knowledge  acquisition  phase,  and 
thus,  had  a  major  impact  on  system  design.) 

The  hurry  factor  Involves  a  combination  of  each  base  preparing 
three  to  four  TAFs  a  day  (see  TABLE  12)  interspliced  with  a  constant  need  to 
brief  aircrews  and  ground  operations  on  enroute  or  base  weather.  In  addition, 
there  are  administrative,  maintenance,  and  training  requirements  for  many 
base  AWS  personnel.  The  hurry  factor  could  cause  a  vital  piece  of  information 
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TABLE  11.  Key  Factors  That  Could  Cause  Fog  Forecast  Error. 


I. 

The  Hurry  Factor 

A. 

Vital  parameter  not  considered 

B. 

Inadequate  preparation  of  the  forecast 

1.  Careless  forecaster  worksheet  preparation 

2.  Lack  of  forecast  continuity 

3.  Collection  of  data  incomplete 

II. 

Increased  Data  Factor 

1. 

Forecaster  misled  by  additional  or  special  observation 
information  after  forecast  time 

III. 

The 

Bust  Fear  Factor 

1. 

Forecaster  hedges  on  forecast  ("sits  on  the  fence") 
for  fear  of  missing  or  busting 

IV. 

Logical  Factor 

1. 

No  way  the  forecast  could  have  logically  been  made 

V. 

Carelessness  Experience  Factor 

1. 

Pure  carelessness  or  lack  of  experience  in  making  a 
fog  forecast. 

i 

i 


TABLE  12. 

Summary  of  TAF  Times  for  the  Selected  Airbases. 

Base 

Times  (Zulu) 

2200 

Seymour  Johnson  (GSB) 

2200 

Fort  Bragg  (FBG) 

NOTE:  TAFS  can  be  amended  as  needed. 
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to  be  missed  or  a  whole  train  of  thought  to  become  discarded.  For  example,  a 
forecaster  may  iirmedi ately  notice  a  severe  weather  threat  due  to  dry  air 
intrusion  at  700  mbar,  but  he/she  (in  turn)  may  skip  over  (due  to  priorities) 
the  fact  that  conditions  later  may  be  favorable  for  fog. 

Therefore,  an  expert  system  could  solve  many  of  these  factors  with 
special  emphasis  on  the  lack  of  time  and  corresponding  quantity  of  resources 
that  a  forecaster  needs  to  prepare  the  TAF. 

2.4  IDENTIFICATION  OF  AVAILABLE  COMPUTER  RESOURCES 

Our  final  step  in  identifying  domain  characteristics  was  to  categorize 
and  then  select  AI  software  that  could  create  the  expert  system. 


The  three  airbases  all  have  Zenith  Z-100  microcomputers  with 
256  K  memory,  monochrome  displays,  dual  disk  drives  and  various  printer 
configurations. 

The  Z-100  is  not  a  directly  PC-compatible  machine,  thus  posing  some 
problems  in  software  selection.  The  following  two  subsections  describe 
efforts  to  identify  the  appropriate  AI  software  to  begin  KBES  development. 


2.4.1  AI/KBES  Development  Software--Overview 


The  purpose  of  this  subsection  is  to  briefly  review  the  various 
expert  system  development  tools  and  to  present  our  strategy  in  selection  of 
the  proper  tool.  Expert  system  development  tools  can  be  viewed  as  software 
packages  that  aid  in  the  creation  of  the  expert  system.  Basically,  there  are 
two  general  software  methods  to  create  an  expert  system: 


•  Computer  languages 


•  Compiled  AI/KBES  shells. 


Purely  Al-oriented  computer  languages  center  around  the  use  of 
languages  called  LISP  (LISt  Processing)  and  PROLOG  (PROgramming  language  for 
LOGic).  Both  languages  compute  with  symbolic  expressions  rather  than 
numbers.  In  fact,  PROLOG  is  more  logic-oriented  than  LISP  and  includes 
declarative  and  procedural  styles  of  programming. 

Other  languages  that  are  not  purely  AI  oriented  but  have  been  used  for 
building  expert  systems  include  FORTRAN,  PASCAL,  and  C.  GEOMET  is  currently,  for 
example,  examining  a  PC  version  of  a  KBES  written  in  PASCAL  that  determines 
enemy  course  of  action.  The  C  language  is  also  now  being  viewed  as  a  very 
popular  Al-oriented  language  primarily  because  of  its  speed  and  reverse 
notational  capabilities. 


A  KBES  shell  is  a  "canned"  software  package  that  aids  in  expert 
system  development  through  input  of  rules  and  knowledge  much  like  Lotus  aids 
in  graph  creation  through  data  input  by  the  user.  Many  shells  now  exist  on 
the  market;  however,  certain  criteria  can  be  established  to  screen  these 
shells  prior  to  deciding  on  the  software. 

2.4.2  Computer  Language  Expert  System  Development  Approaches 

An  AI  computer  language  approach  in  expert  system  development  has 
several  advantages  over  using  a  shell  approach: 

•  Flexibility  in  program  development 

•  Flexibility  in  editing/debugging 

•  Built-in  functions  (called  primitives). 

The  greatest  advantage  in  using  this  approach  is  in  the  flexibility  of  the 
overall  system  or  module  development. 

LISP,  which  was  invented  In  1954  at  MIT,  represents  data  as  list- 
linked  structures.  Almost  everything  in  LISP  revolves  around  developing 
these  list  structures,  but  LISP  (unlike  FORTRAN)  has  never  been  standardized. 
This  is  because  there  are  only  a  few  basic  LISP  functions  and  the  programmers 
can  create  any  number  of  higher  level  functions  using  the  small  number  of 
basic  functions. 

The  flexibility  of  the  LISP  language  has  led  to  programming 
advancements  designed  to  address  a  user-specific  problem  or  special  computing 
environment.  Thus  many  variations  of  LISP  exist  (for  both  PC  and  mainframe 
use)  including: 

•  IQLISP  (PC  and  mainframe) 

•  Common  LISP  (IBM  PC-AT,  1M  RAM,  large  memory, 
and  mainframe) 

•  TLCLISP  (PC  and  mainframe) 

•  8YS0LISP  (mainframe) 

•  InterLISP  (PC  and  mainframe). 

Basically  LISP  has  two  data  structures,  atoms  and  lists.  An  atom 
is  an  object  that  cannot  be  broken  down  any  further,  thus,  it  is  typically  a 
name  or  number.  Lists  in  turn  are  composed  of  atoms  (or  in  some  cases  other 
lists).  Recursive  procedures,  involving  searching  through  lists  until  the 
unknown  atom  is  found  or  until  a  question  needs  to  be  asked  of  the  user,  is 
also  a  highly  desirable  feature  of  LISP. 


PROLOG,  which  was  invented  in  France  in  1972  and  is  the  principal 
Japanese  Fifth  Generation  and  European  AI  language,  has  seen  a  recent  (mi d - 1 985 
to  present)  surge  in  use  in  the  United  States.  PROLOG  is  based  on  the 
concept  of  predicate  calculus.  To  invoke  predicate  calculus  reasoning  a 
programmer  first  specifies  facts  about  objects  and  rel at ionsh ips .  He/she 
then  establishes  linkages  regarding  the  objects  and  relationships.  The 
programmer  states  the  facts  and  PROLOG  structuring  determines  whether  any 
specific  conclusion  can  be  deduced  from  the  facts.  PROLOG  is  therefore  the 
closest  language  that  represents  true  logical  deduction.  PROLOG  also  uses  a 
form  of  "backward  chaining"  where  it  searches  for  a  match  of  conditions  that 
meet  the  conclusion.  Primatives  (inherent  functions)  and  atoms  also  exist 
in  PROLOG. 


Several  PC  versions  of  PROLOG  nxist  including: 

•  Micro-PROLOG 

•  PROLOG  1/2 

•  MPROLOG 

•  Arity  PROLOG. 

One  of  the  widely  used  PC-oriented  PROLOGS  is  called  Arity  PROLOG. 
Arity  features  include: 

•  String  support 

•  Speed 

•  Primitive  resources 

•  Linkages  to  external  programs. 

The  string  text  support  provided  by  the  Arity  compiler  and  inter¬ 
preter  goes  beyond  the  atom  level.  This  means  that  phrases  or  concatena¬ 
tions  can  be  supported.  Arity  is  the  fastest  micro-based  PROLOG  compiler 
available  and  has  the  ability  to  handle  arithmetic,  floating  point,  and 
computational  quantities.  Arity  also  has  the  inherent  capability  of  over  150 
primitives,  which  in  itself  is  an  aid  to  a  programmer.  Arity  also  provides 
for  linkages  to  other  programming  languages  such  as  0,  Assembly,  FORTRAN, 
PASCAL,  or  even  LISP. 

A  separate  specialized  expert  system  development  package  that  is 
included  with  the  Arity  PROLOG  compiler  and  interpreter  allows  the  creation 
of  modules  either  using  a  frames-based  architecture  or  a  rules-based  architec¬ 
ture.  The  rules-based  architecture  is  geared,  however,  not  toward  the  true 
if-then  structure  but  to  deriving  rule  values  from  other  frames.  Frames  and 
rules  will  be  further  explained  in  the  knowledge  representation  sector.  The 


-41- 


ilii. 


Arity  expert  system  development  package  requires  the  PROLOG  compiler  and 
interpreter.  Unfortunately,  as  in  most  PROLOG-compi led  systems,  Arity  is 
memory  hungry.  A  minimum  of  512  K  memory  is  required  on  an  IBM  PC-AT  with 
640  K  memory  recommended. 

2.4.3  The  KBES  Shell  (Tool)  Expert  System  Development  Approach 

As  part  of  this  and  other  AI  efforts,  we  have  and  continue  to 
update  market  surveys  of  current  software  shells.  Several  larger  shells 
(sometimes  called  "tools"),  which  require  discussion,  exist  in  the  marketplace 
These  larger  shells  or  "tools"  are: 

•  Knowledge  Engineering  System  (KES) 

•  Automated  Reasoning  Tool  (ART)  and  other  "large  system" 
shel 1 s 

•  Knowledge  Engineering  Environment  (KEE). 

KES 

KES  contains  backward  chaining,  but  no  forward  chaining,  and  has 
hypothetical  reasoning  and  object  description.  KES  cannot  be  imbedded  into 
other  systems.  KES  is  sold  in  pieces  but  generally  does  not  include  an 
editor,  which  is  a  major  user-friendly  drawback.  KES  is  really  like  a  batch 
file  expert  system  where  the  knowledge  base  is  written  using  (for  example) 
WORDSTAR  and  then  compiled  in  KES.  This  really  slows  down  and  complicates 
the  development  process.  KES  is  compiled  in  various  forms  of  LISP. 

ART  and  Other  "Large  System"  Shells 

ART  is  available  on  LISP  computers  (Symbolics,  Xerox,  etc.).  ART 
is  more  oriented  toward  a  primary  expert  system  tool  and  can  perform  backward 
and  forward  chaining.  ART  consists  of  four  components:  a  knowledge  language, 
a  compiler,  an  appller,  and  a  development  environment. 

The  drawbacks  in  using  ART  for  this  application  are  twofold: 

(1)  ART  cannot  be  easily  transported  from  its  resident  LISP  machine  down  to  a 
PC  and  (2)  ART  Is  too  knowledge-engineer-oriented  for  the  proof-of-concept 
task  requirement. 

The  downloading  of  expert  systems  created  by  ART  (or  other  similar 
creation  language  shells  such  as  0PS5  or  ROSIE)  on  larger  machines  to  a  PC  is 
a  difficult  and  laborious  task.  The  steps  involved  are: 

1.  Scale  down  and  recompile  rules  on  the  large  system 

2.  Download  (modem  hookup)  to  PC 


3.  Recompile  rules  with  a  LISP  compiler  on  PC 
(and  hope  for  the  best). 


Each  step  is  filled  with  challenges  such  as  how  to  scale  down 
object-attribute  values  (or  ART  quantifiers)  for  transport  without  losing  the 
larger  system  hierarchical  intent.  Each  step  also  takes  a  considerable  amount 
of  time,  which  can  be  better  used  to  refine  a  resident  (originally  based) 

PC  system. 


Also,  many  larger  or  specialized  machine  expert  system  tools 
are  too  purely  Al-oriented.  Certainly  arguments  can  be  made  that  changes  to 
large  meteorological  automated  system  be  made  on  one  large  machine  and 
then  sent  to  the  base  PCs;  however,  each  change  to  the  PC  version  coming 
down  from  the  larger  machine  must  proceed  through  the  three  difficult  and 
time-consuming  steps  above.  This  is  not  to  say  that  this  can't  be  done,  just 
that  it  goes  beyond  the  purpose  of  a  proof-of -concept .  In  addition,  any 
minor  (three-  to  five-rule)  base-specific  changes  must  be  sent  through  the 
large  machine,  and  hence  through  the  three  steps  rather  than  the  easier 
route,  which  could  allow  the  base  itself  (or  the  knowledge  engineering  team) 
to  quickly  make  changes  on  the  spot.  Extensive  training  is  necessary  to  use 
ART. 


KEE 

Another  tool  is  Knowledge  Engineering  Environment  (KEE),  which  is  a 
forward  and  backward  chaining  system  with  hypothetical  reasoning.  KEE  also 
provides  full  object  description  using  frames.  KEE  has  PC  application 
problems  similar  to  ART  because  it  must  be  transported  from  specialized  LISP 
machines  such  as  the  Xerox  1100  or  Symbolics  3600  series  down  to  the  PC 
level.  KEE  is  also  very  complicated  and  extensive  training  is  necessary. 

Many  smaller  shells  specifically  for  PC  application  exist  and  are 
described  below.  These  shells  are: 

•  The  Intelligent  Machine  Model  (TIMM) 

t  Rulemaster 

•  Expert  Ease 

t  Personal  Consultant/Plus 

•  Insight  2  Plus 

•  Ml. 


TIMM 


TIMM  has  forward  chaining,  but  no  backward  chaining,  which  puts  it 
at  a  great  disadvantage  for  meteorological  users.  Objects  are  described  in 
terms  of  attributes  of  a  single  problem.  The  base  language  is  FORTRAN  77, 
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which  makes  TIMM  much  slower  than  other  systems.  TIMM  is  more  examples 
based,  where  each  set  of  examples  forms  a  matrix  and  each  matrix  is  a  rule. 

This  approach  could  cause  problems  in  knowledge  representation  and  acquisition. 
A  similar  system  called  K I  BASE  can  run  on  a  Symbolics  (larger  machine)  or  a 
PC,  but  has  the  same  problems  as  TIMM  and  the  same  inherent  transfer  problems 
as  ART. 

Rulemaster 

Rulemaster  is  a  simplified  form  of  machine  intelligence  because  the 
only  logic  is  in  a  decision-tree  format.  Forward  chaining  is  not  supported. 
Rulemaster  was  originally  imported  from  England  where  decision-tree  induction 
was  a  popular  format  in  the  late  1970's.  Unfortunately,  the  decision-tree- 
examples  format  is  the  only  problem  solver  provided  and  this  could  pose 
problems  for  our  application,  particularly  in  the  synoptic  area,  where  much 
time  could  be  wasted  in  determining  proper  examples  and  then  finding  out  that 
the  examples  selected  are  incomplete.  A  second  negative  factor  involves  the 
use  of  the  radial  language,  which  requires  completion  each  time  code  is 
changed.  These  two  factors  have  been  a  major  drawback  in  the  knowledge 
acquisition  and  consequent  representation  process  of  Rulemaster  ever  since 
its  importation  into  the  United  States. 

Expert  Ease 

Expert  Ease  has  no  backward  or  forward  chaining  but  operates  like 
Rulemaster  in  that  it  is  a  simplistic,  decision-tree-examples  problem  solver. 
Expert  Ease  recently  failed  a  NASA  sample  expert  system  development  problem 
and  is  viewed  by  tne  AI  community  as  a  simplistic  first-time  expert  system 
learning  tool,  which  is  not  appropriate  for  full-scale  expert  system  develop¬ 
ment  . 

Personal  Consultant/Plus 

This  system  uses  forward  and  backward  chaining  in  both  system 
versions  (regular  and  plus)  and  uses  the  object-attribute-value  scheme  of 
(described  in  Section  3.0)  representing  rules.  Personal  Consultant,  however, 
is  not  very  user  friendly  when  producing  rules  and  even  though  the  system  has 
facilities  for  the  "unknown"  response  to  a  question,  reasoning  with  "unknown" 
is  somewhat  hampered  by  the  internal  program  structure. 


Insight  2  Plus 

Insight  represents  rules  in  either  attribute  value  or  object- 
attribute-value  triplets.  Insight  2  has  recently  been  updated  to  Insight  2 
Plus;  Insight  3  is  the  VAX  version,  which  will  be  available  later  in  1987. 
Insight  2  Plus  is  a  goal-driven  shell  only  that  has  had  several  problems 
associated  with  memory  requirements  during  various  recent  NASA  tests. 
Insight  also  does  not  allow  for  efficient  arithmetic  computation  within  the 
shell  itself. 


Ml  operates  on  an  IBM  PC/XT/AT  using  a  minimum  of  384K  bytes  of 
memory  (512K  memory  is  recommenced).  Ml  is  a  highly  sophisticated  KBES 
development  tool  that  was  originally  compiled  in  PROLOG/1  but  is  now  available  in 
compiled  C.  The  Ml  shell  is  based  on  backward  chaining;  the  shell  does  not 
have  a  true  forward-chaining  capability. 

Ml  is  particularly  sensitive  to  the  order  in  which  rules  are  placed 
into  the  shell.  It  is  also  sensitive  to  the  use  of  metaknowledge  (knowledge 
within  knowledge). 

Other  features  of  Ml  include: 

•  Reason  explanation  (an  advantage  in  meteorological 
appl ications) 

•  Arithmetic  computational  capabilities 

•  Degrees  of  certainty  or  uncertainty 

•  Multiple  window  displays  (aids  user/developer 
considerably) . 

Given  all  the  advantages  of  Ml  outlined  above,  there  appears  to  be  three 
disadvantages:  blackboard,  price,  and  training. 

Ml  has  no  direct  provision  for  blackboard  space.  Instead,  the 
programmer  is  required  to  write  linkage  programs  in  another  language  and  then 
transfer  qualifers  among  the  modules.  This  is  quite  cumbersome. 

The  Ml  base  price  of  $5,000  is  also  competitively  high  when  com¬ 
pared  to  other  shells.  The  $5,000  price  includes  five  user  copies  of  the 
Ml-created  expert  system  (but  not  the  generator).  Each  additional  user  copy 
costs  $500  (up  to  10),  then  the  cost  drops  to  $250. 

Finally,  Ml  requires  extensive  vendor-supported  training  that  car 
be  tedious  and  very  time  consuming. 


2.4.4  Selection  of  Our  Shell 

Several  criteria  were  used  to  select  the  shell  for  this  effort. 

First,  the  tool  or  large  shell  approach  was  ruled  out  due  to  the  limitations 
of  the  Z-100  computer.  Tools  usually  need  dedicated  (and  expensive)  AI 
machines  to  run  on.  We  also  ruled  out  the  use  of  a  language  as  a  proof-of- 
concept  software  development  mechanism  because  of  the  time  involved  in  programming 
It  was  felt  that  time  could  be  better  used  developing  the  knowledge. 


Thus,  we  were  left  with  the  shell  approach  by  virtue  of  their 
flexibility  in  running  on  PC -compatible  machines.  AWS  detachments  are  in  the 
process  of  upgrading  their  Z-lOOs  to  Z-248s  or  IBM  PC-compatible  Zenith 
PCs,  thus,  we  selected  a  shell  that  can  run  on  a  PC-compatible  machine.  We 
also  wanted  the  AWS  personnel  to  evaluate  the  system  now,  so  IBM  PC  clones 
were  provided  to  each  base  to  test  Zeus. 

The  key  criteria  (or  questions)  we  used  for  selection  of  a  KBES 
shell  are  as  follows: 

KBES  Shell  Criteria 

1.  Does  the  shell  have  computational  capability?  This 

is  especially  important  when  deriving  meteorologically 
important  parameters  such  as  potential  temperature, 
lapse  rate,  etc. 

2.  Does  the  shell  allow  probability  calculation?  This 
is  especially  important  for  our  purposes  because  the 
end  KBES  result  is  that  the  base  weather  officer  will 
examine  a  listing  of  the  probability  of  visibility 
less  than  certain  ranges  (i.e.,  probability  of  less 
than  a  half-mile,  1  mi ,  3  mi ,  etc.) 

3.  Can  the  shell  link  to  external  programs/monitors? 

This  feature  is  extremely  important  because  it  allows 
the  KBES  to  pause  and  obtain  necessary  information 
from  an  external  program  in  BASIC,  FORTRAN,  dBASE 
III,  LOTUS,  etc.  This  information  could  be  based  on 
an  initial  KBES  user  input  variable  (i.e.,  humidity, 
temperature)  and  an  external  program  result  (dew¬ 
point),  which  is  then  transferred  back  to  the  KBES  for 
further  use.  Linkage  to  meteorological  monitors  is 
also  noteworthly  (i.e.,  windspeed  could  be  auto¬ 
matically  input  from  the  sensors  instead  of  keyed  in 
by  the  base  meteorologist). 

4.  Does  the  shell  permit  the  KBES  to  run  in  a  reasonable 
time?  KBES  FORTRAN-  and  PASCAL-based  shells  run  much 
slower' than  LISP  and  C  language  shells.  C  language 
shells  are  faster  than  LISP  shells. 

5.  Is  the  shell  user-friendly?  This  is  an  extremely 
important  consideration  and  involves  display  tech¬ 
niques  and  help  facilities. 


6.  Is  the  shell  price  competitive  and  capable  of  running 
on  a  standard  PC?  The  price  of  shells  run  anywhere 
from  $15,000  down  to  $100. 

The  shell  called  "EXSYS"  and  developed  by  EXSYS,  Inc.  of  New  Mexico, 
was  selected  because  it  met  all  of  the  criteria  previously  outlined.  The 
EXSYS  shell  (VER  3.0)  was  developed  in  1983,  and  is  now  used  by  over  1600 
groups.  Polaroid  uses  EXSYS,  for  example,  in  many  of  its  decision-making 
process  applications.  The  Dupont  Corporation  has  also  used  EXSYS  as  part  of 
its  extensive  expert  system  development.  Other  EXSYS  application  areas 
include  medicine,  agriculture,  and  construction. 

EXSYS  more  than  meets  all  of  the  criteria  outlined  above  and  also 
provides  a  "what-if"  feature  that  allows  meteorologists  to  perform 
sensitivity  analyses  on  key  questions  asked  by  the  KBES.  Thus,  GEOMET 
meteorologists  will  be  able  to  easily  see  what  effect  changing  one  or  more  of 
the  user  answers  will  have  on  the  conclusion. 

The  EXSYS  shell  attributes  (in  addition  to  meeting  the  criteria 
mentioned)  are  as  follows: 


•  Minimum  of  256  K  required  =  700  rules.  Every  64  K 
over  256  K  =  700  rules.'  We  can  run  EXSYS  on  one  of 
our  640  K  PC-AT,  which  will  handle  about  5,000  rules. 
The  base  I8M  clone  PCs  (640K)  can  also  handle  5,000 
rules. 

t  Arithmetic,  trigonometric,  log,  and  square  root 

functions  are  supported. 

t  Backward-type  chaining  is  supported  allowing  large 

problems  to  be  broken  down  into  smaller  ones. 

0  Forward-type  chaining  is  also  supported  allowing  for 
more  intensive  data-driven  appl icat ions. 

0  Report  generation  procedures  (i.e.,  how  EXSYS  arrived 
at  a  visibility  forecast  of  less  than  1  mi  and  a 
simple  end  result  format). 

0  English  text,  menu  selection,  algebraic  expressions, 
and  color  supported. 

0  C  language  based  for  speed. 

0  Unknown  accepted  as  an  answer. 

0  VAX  compatabi 1 ity  (can  upload  and  run  on  a  VAX  if 
need  be) . 


The  EXSYS  shell  consists  of  two  programs:  Rule  Editor  and  Runtime. 
The  Rule  Editor  can  be  used,  for  example,  to  create,  edit,  or  delete  meteoro¬ 
logical  rules  (i.e.,  modify  the  knowledge  base).  The  Runtime  program  runs 
the  rules  created.  Thus,  different  module-specific  Runtime  programs  can  be 
created  and  yet  maintenance  of  a  general  meteorological  expert  system 
architecture  is  possible. 

EXSYS  also  provides  a  "what-if"  feature  that  can  allow  a  knowledge 
engineering  team  to  perform  sensitivity  analyses  on  key  questions  asked  by 
the  KBES.  Therefore,  the  team  has  the  capability  to  easily  see  what  effect 
changing  one  or  more  of  the  user  answers  will  have  on  the  conclusion. 

EXSYS  also  has  a  blackboard  feature  that  allows  for  rapid  transfer 
of  knowledge  or  data  between  system  modules.  EXSYS,  which  is  rapidly  growing, 
has  an  agressive,  ongoing  corporate  R&D  program  to  provide  user  interface 
aids  such  as  lookup  table  linkages  and  other  desktop  functions  directly  into 
EXSYS,  making  EXSYS  one  of  the  most  efficient  at  linking  an  external  data 
base  with  a  shel 1 . 


Section  3.0 

KNOWLEDGE  REPRESENTATION 


The  problem  of  representing  knowledge,  and  particularly  temporal 
knowledge,  arises  in  a  wide  range  of  disciplines,  including  computer  science, 
philosophy,  logistics,  and  psychology.  This  section  describes  our  selection  of 
meteorological  knowledge  representation  techniques. 

3.1  BRIEF  REVIEW  OF  POSSIBLE  KNOWLEDGE  REPRESENTATION  SCHEMES 

Knowledge  within  an  expert  system  can  be  represented  using  a  variety 
of  techniques.  The  two  major  architectural  mechanisms  for  representing 
knowledge  are: 

•  Forward  Chaining 

•  Backward  Chaining. 

Under  forward  chaining,  the  entire  process  is  data  driven  and  the 
various  rulepaths  within  the  expert  system  examine  the  available  data  and  try 
to  test  any  data-specif ic  hypothesis  to  acquire  more  facts  or  knowledge  about 
a  situation.  This  architecture  could  be  most  useful  in  the  emergency  threat 
environmental  area  (i.e.,  dispersion  of  a  toxic  gas)  or  other  areas  where 
final  goals  are  not  clear. 

Under  backward  chaining,  the  rulepaths  are  oriented  toward  a  common 
main  goal.  This  goal  is  achievable  if  the  rules  satisfy  various  subgoals. 

In  other  words,  the  conclusions  are  already  in  the  system  and  the  job  of  the 
system  is  to  test  to  see  if  those  conclusions  can  be  proved  with  the  knowledge 
the  system  has  within  itself. 

Either  representation  architecture  will  work  in  the  meteorological 
area;  however,  it  is  our  experience  that  backward  chaining  is  much  more 
readily  convenient  given  the  natural  structure  of  environmental  problem 
areas,  and  "‘rticularly  meteorology,  toward  goal  orientation.  An  example  of 
meteoroloa’  '  goals  include  categories  of  low  visibility  (in  our  case)  or 
could  inc  items  such  as  the  rain  or  snow  forecast  goals,  windspeed  goals, 
or  tempe’  ,,e  goals. 
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The  backward-chaining  approach  can  be  a  very  powerful  tool  within 
a  specific  meteorological  area  such  as  mesoscale  meteorology  where  subgoals 
can  be  established  and  the  expert  system  can  review  required  subgoals  with 
the  user  indicating  to  the  user  where  and  how  subgoals  were  derived. 


The  representation  of  knowledge  in  the  expert  system  under  backward 
chaining  still  takes  the  form  of  rules.  These  rules  can  be  represented  in 
four  different  methods: 


Object-Attribute-Value  (OAV  triplets  or  Attribute- 
Value  (AV)  pairs) 


Frames 


Semantic  networks 


Logical  expressions. 


The  OAV  method  describes  objects  as  either  conceptual  or  physical 
quantities,  attributes  as  a  characteristic  of  the  object,  and  the  value  as 
the  specific  nature  of  the  attribute  as  indicated  below: 


Example  of  OAV  Triplet 


Surface  Wind 

t 

OBJECT 


Greater  Less 

than  35 3°  and  than  120*  Trajectory  Sufficient  90  Percent 

♦  t  t 

ATTRIBUTE  RESULT  VALUE 


The  relationships  of  OAVs  can  be  recorded  within  a  backward- 
chained  system  to  reflect  a  "dynamic"  knowledge  change  (changes  every  time 
system  is  run)  or  as  "state"  knowledge  (same  every  time).  In  this  example, 
the  object  in  question  is  the  surface  wind;  the  attribute  refers  to  the 
characteristic  of  that  wind.  The  OAV  approach  can  also  have  a  certainty 
factor  attached  to  it,  such  as  a  measure  of  confidence  that  the  trajectory  is 
sufficient  (i.e.,  the  value).  Certainty  factors  represent  the  degree  to 
which  the  OAV  standard  rule  is  true. 


The  AV  method  is  similar  to  the  OAV  method  except  that  multiple 
objects,  such  as  types  of  wind  (i.e.,  gradient,  850  mbar)  cannot  be  represented 
properly.  GEOMET  has  used  OAV  triplets  in  several  of  its  expert  system 
applications,  but  has  not  used  very  many  AV  pairs. 


Frames  provide  a  different  means  of  structuring  knowledge.  The 
idea  of  a  frame  has  been  introduced  in  the  AI  literature  in  1974  by  Minsky 
(20)  as  a  slot  concept.  Each  object  (such  as  the  surface  wind)  has  a  series 
of  slots.  Slots  can  represent  properties  associated  with  the  wind  (or 
default  values  if  information  is  not  available),  and  various  "inheritance" 
features  that  can  lead  to  other  frames  in  the  path  leading  up  to  the  goal  or 
subgoal . 


Each  slot  within  a  frame  can  have  various  procedures  attached  or 
triggered  by  the  slot  as  shown  in  Fig.  13.  Typical  examples  include: 

If  needed  procedures:  The  slot  is  empty--rules  execute  when 

knowledge  is  needed  for  the  slot 

If  added  procedures:  Rules  execute  when  new  knowledge  is 

placed  in  the  slot 

If  removed  procedures:  Rules  run  when  knowledge  is  deleted 

from  the  slot. 

The  frames  figure  indicates  the  same  basic  OAV  triplet  as  before,  but  now  the 
triplet  is  imbedded  within  a  wind  frame. 

The  slot  and  frame  procedure  is  particularly  useful  in  organizing 
large  knowledge  data  sets. 

Semantic  networks  involve  nodes  that  represent  objects  and  various 
direct  linkages  that  relate  to  the  nodes  by  definition  or  direct  action. 
GEOMET  has  used  one  semantic  network  to  represent  certain  terrain  features; 
however,  the  difficulty  in  semantic  networks  is  in  their  broadness.  OAV 
triplets  and  other  methods  are  much  more  specific. 

Logical  expressions  refer  to  propositions  such  as  AND,  NOT,  and 
OR.  These  expressions  are  extremely  powerful  in  an  OAV  or  frames  format.  Fo 
example,  two  wind  rules  sharing  the  same  values  can  be  combined  into  one 
rule  using  AND  or  two  other  rules  can  be  declared  as  conditional  using 
OR. 

3.2  SELECTION  OF  REPRESENTATION  SCHEME 


From  the  conceptual  models  developed  in  Section  2.0  during  the 
identification  of  the  problem,  it  became  readily  apparent  that  a  backward¬ 
chaining,  goal -directed,  rule-driven,  proof-of-concept  system  was  required 
for  several  reasons: 

1. 

A  goal -directed  system  put  definite  bounds  on  the 
base  meteorological  categories. 

2. 

Forecasters  tend  to  think  in  terms  of  OAV  triplets 
(with  less  emphasis  on  confidence  values  assigned  to 
the  triplet). 

3. 

Four  clear  requirements  were  identified  for  any  fog 
forecast . 

£ 


Characteristic  1  Direction 


L- Noncritical 


If  Added  and  Removed 
Procedures/Rules 


r-Critical 


Characteristic  2  Speed 


If  Added  and  Removed 
Procedures  and  Rules 


>--Noncritical 


r- If  Needed  Procedures 


Characteristic  3  Temporal 
History 


(link  to  another  frame) 


■If  Changed 


Figure  13.  Typical  organization  of  meteorological  frame. 


First,  a  goal-directed  system  as  a  proof-of-concept  made  a  lot  of 
sense  given  the  base  TAF  requirements  of  parameter-bound  or  category  forecasts 
of  low  visibility  (<1,  1  to  3,  3+  mi).  Reaching  these  goals  can  be  easily 
structured  in  any  backward  chaining  format,  but  particularly  in  EXSYS. 

The  conceptual  models  developed  during  the  identification  phase 
also  clearly  indicated  fog  breakpoints,  such  as  advection  and  radiation,  along 
with  attendent  character i st ics  of  each,  thus,  promoting  the  idea  of  subgoals 
under  each  area.  Second,  we  found  in  the  TFRNs,  the  other  literature,  and 
by  interview  that  forecasters  inherently  think  in  terms  of  if-then-else 
rules,  thus,  an  expert  system  that  can  draw  on  that  type  of  structure  can 
become  a  powerful  tool. 

Finally,  as  a  result  of  close  examination  of  the  conceptual 
models,  it  was  found  that  in  any  given  (fog)  forecast,  five  basic  things 
could  happen: 

•  Forecast  degradation  from  current  visibility  >  3 
conditions 

•  Forecast  degradation  from  an  existing  condition 

•  Forecast  improvement 

•  Forecast  improvement  to  a  better  condition  (but  still 

below  criteria) 

•  Forecast  persistance. 

In  the  first  case,  visibility  may  be  greater  than  7  mi  and  fore¬ 
cast  to  degrade  to  2  mi.  Similarly,  visibility  could  be  2  mi  and  drop  to 
1/2  mi  (degradation  from  existing  condition).  In  the  third  and  fourth  cases, 
visibility  could  improve  from  1/2  to  7+  mi  or  improve  from  1/2  to  2  mi  and  in 
the  fifth  case,  visibility  could  persist  at  a  low  1/2  mi  for  several  hours. 

From  the  conceptual  models  and  the  five  basic  ideas  regarding 
overall  structure  outlined  above,  a  preliminary  system  design  was  developed. 
This  design  reflects  the  forecast  degradation  case  and  depicts  the  initial 
ideas  behind  representing  the  knowledge.  Subgoals  derived  from  the  conceptual 
model  receive  information  from  a  synoptic  module  and  then  follow  rulepaths  to 
a  major  or  minor  goal  depending  on  how  various  rules  are  executed,  or  in  AI 
terminology  "fired." 

It  was  becoming  increasingly  obvious  at  this  time  that  various 
conditions  needed  to  be  met  to  establish  the  subgoal  (or  goal)  as  being 
valid.  This  now  meant  that  all  rules  would  relate  to  "conditions."  This,  in 
turn,  impacted  on  knowledge  acquisition  because  interview  discussions 
were  oriented  along  the  lines  of  obtaining  necessary  and  sufficient  conditions 
to  satisfy  the  go^ls  or  subgoals  through  rulepath  formulation. 


We  also  began  to  realize  with  the  initial  structure  that  clock  time 
would  become  a  critical  aspect  throughout  the  expert  system.  We  consequently 
introduced  clock  time  in  a  subsequent  design  review  described  in  the  next 
subsection. 

3.3  LINKING  KNOWLEDGE  REPRESENTATION  TO  THE  METEOROLOGY 

3.3.1  Overall  Structure 

It  is  important  to  reiterate  that  knowledge  representation  and 
acquisition  go  hand-in-hand  during  expert  system  development.  One  cannot 
acquire  domain  expertise  through  interviews  with  an  expert  without  first  at 
least  knowing  how  that  knowledge  will  be  represented.  (That  is  why  the 
identification  phase  is  so  critical  to  the  success  of  the  project.) 

Our  initial  representation  scheme  was  consequently  modified  by  the 
interviews.  The  new  (and  final)  scheme  is  given  in  Fig.  14.  Solid  lines 
represent  rulepaths  throughout  the  system. 

The  structure  revolves  around  the  user  entering  the  current  observa¬ 
tion  with  a  natural  branch  being  triggered  by  this  observation  (i.e.,  greater 
than  or  less  than  3  mi--a  common  base  TAF  checkoff  point).  The  system  then 
decides  whether  one  of  the  four  basic  items  of  degradation  (from  existing), 
degradation  (from  nonexisting),  improvement  (to  existing),  improvement  or 
persistence  will  occur.  Should  the  current  visibility  be  greater  than  3  mi, 
then  the  two  possible  solutions  to  the  forecast  are  degradation  below  3  mi  or 
persistence  (remaining  good)  on  the  other  side,  if  visibility  is  below  3  mi, 
then  any  one  of  the  four  solutions  could  occur.  This  key  breakpoint  is 
graphically  illustrated  in  Fig.  15. 

After  the  controller  determines  the  proper  path,  Zeus  then  proceeds 
to  assess  each  module  in  terms  of  whether  necessary  and  sufficient  conditions 
are  met  (i.e.,  the  subgoals).  The  following  subsections  describe  each  module 
drawing  on  the  knowledge  obtained  from  development  of  the  conceptual  models. 

3.3.2  The  Advection  Module 

The  advection  module  for  each  base  is  very  similar  in  that  its 
major  goal  is  to  determine  whether  "Atlantic  Flow"  (AF)  is  sufficient.  The 
need  for  an  oceanic  trajectory  has  been  discussed  under  the  advection  concep¬ 
tual  model  presented  earlier. 

The  advection  module,  independent  of  the  synoptic  module,  determines 
first  whether  AF  exists;  then  executes  rules  to  determine  surface  and  aloft 
moisture  values,  and  finally  integrates  synoptic  rules.  The  basic  outline  of 
the  module  appears  in  Fig.  16. 


User  Output : 
Advlct  And 

V 1 s 1 b 1 1 Ity  C*t«9ories 


f'jurt  i4.  flrn)  Ztui  %ntm  dgsign. 


;  No 

i 


Control ler 

Vis  <  3 
Miles 


Yes 


Degradation 


Per si stence 


Degradation 

Further 


Improve 


Per si st 
at 

Current  Level 


Critical  Surface 
Wind  Sector 
Analysis 


Condition  1 


NOTE:  Any  negatives  associated  with  any  of  these  three  condition  boxes  indicates 
no  AF  at  the  moment. 


The  rules  are  divided  into  three  key  areas  that  are  organized  as 
conditions  1,  2,  and  3  (or  subgoals  1,  2,  3).  In  the  first  area,  critical 
surface  wind  sectors  were  established  for  each  base.  These  sectors  reflected 
surface  flow  off  the  Atlantic  and  are  based  on  a  combination  of  TFRN  informa¬ 
tion  and  interviews.  The  second  area  involved  a  simplified  comparison  of 
dewpoint  depression  between  nearby  stations  to  obtain  an  idea  of  how  much 
moisture  was  being  advected  into  the  area.  At  Dover,  the  70  percent  RAFS  RH 
line  was  also  used,  although  this  parameter  was  not  used  at  Simmons  or 
Seymour  Johnson. 

The  third  area  involved  rules  designed  to  use  the  observed  and  FOUS 
predicted  against  both  the  vertical  profiles  of  wind  direction  and  speed. 

This  check  (called  condition  3)  is  critical  because  many  fog  situations  as 
shown  in  the  conceptual  model  discussion  and  later  borne  out  by  interviews 
are  greatly  influenced  by  the  boundary-! ayer  wind  structure. 

If  all  three  conditions  are  met,  then  AF  exists.  Should  any  one  of 
the  conditions  not  be  met,  then  AF  does  not  exist  and  the  program  does  one  of 
two  things  (See  Fig.  17).  It  searches  to  see  if  AF  is  possible  at  a  future 
time  or  it  goes  to  the  radiation  module.  The  advection  module  uses  information 
from  the  synoptic  module  to  confirm  the  AF  existence  and  also  uses  synoptic 
information  to  inform  users  of  the  possibility  of  AF  at  a  later  time. 

The  above  discussion  reflects  the  controller  selecting  the  route 
referring  to  visibility  greater  than  3.  Should  visibility  currently  be  less 
than  3  mi,  then  the  major  function  of  the  advection  module  is  to  determine 
whether  the  present  fog  is  advective  in  nature.  If  so,  then  the  module 
feeds  the  information  to  a  breakout  routine. 

The  breakout  routine  follows  the  conceptual  dissipation  model 
introduced  in  Section  2.0,  however,  we  chose  to  integrate  the  rules  into  the 
advection  and  radiation  modules  because  each  fog  is  treated  slightly  differently 
in  terms  of  dissipation.  One  interesting  feature  of  the  breakout  routine  is 
the  use  of  Pilot  Reports  (PIREPS),  a  possible  source  of  information  on  cloud 
heights  and  thickness.  A  redundancy  feature  has  been  built  into  the  P I  REP 
question  such  that  if  PIREPs  are  not  available,  then  the  user  is  prompted  for 
satellite  information.  Other  satellite-based  assessments  are  made  using 
observations  based  on  rate  of  burnoff  from  the  edges  of  an  advection  (or 
radiation  fog  area).  Fig.  18,  in  summary  form,  indicates  how  the  advection 
module  reacts  to  visibil ity  below  the  3-mi  threshold. 

Some  of  the  problem  areas  that  were  listed  in  the  conceptual  model 
discussion  were  solved  by  knowledge  acquisition  and  structured  as  part  of 
representing  the  knowledge. 
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Figure  18.  Advection  module:  illustration  of 
visibiTTty  <  3  mi  currently. 


In  sunmary,  the  advection  module  is  run  after  the  controller 
determines  the  proper  pathway  to  take  based  on  current  visibility.  The 
advection  module  can: 

•  Determine  whether  Atlantic  flow  exists 

•  Supplement  Atlantic  flow  possibility  assessments 
from  the  synoptic  module 

•  Determine  persistence,  improvement,  or  degradation  of 
visibility  based  on  information  from  the  synoptic  module 
or  from  the  breakout  routine. 

The  early  use  of  the  conceptual  model  greatly  aided  in  the  "fleshing  out"  of 
the  advection  module  during  the  knowledge  acquisition  phase. 

3.3.3  The  Radiation  Module 


The  radiation  module  has,  as  its  primary  function,  to  determine 
whether  radiation-induced  fog  will  occur.  A  key  factor  involved  in  radiation 
fog  formation  is  a  nearby  h i gh-pressure  center  or  ridge.  Thus,  the  module 
relies  on  information  from  the  synoptic  module  to  make  an  initial  judgment 
of  radiation  fog. 

Three  conditions  were  established  for  radiation  fog,  one  of  these 
conditions  is  implied.  The  conditions  are: 

•  Condition  1:  radiation  fog  possible 

•  Condition  2:  special  condition  for  radiation  fog  behind 

warmfronts 

•  Implied  Condition  3:  no  radiation  fog. 

Under  condition  I,  the  radiation  module  uses  input  from  the 
synoptic  module  such  as  positioning  of  the  high  to  reach  conclusions  on  the 
possible  radiation  fog  conditions.  After  passing  this  first  screening,  the 
module  then  basically  follows  the  outline  presented  in  the  conceptual  model 
as  shown  in  Fig.  19. 

The  assessment  of  clear  skies  is  based  on  the  current  and  forecasted 
local  sky  conditions.  The  module  can  take  up  to  scattered  cloud  conditions; 
however,  should  broken  conditions  occur,  messages  are  displayed  indicating 
that  potential  cloudiness  could  eliminate  the  chance  of  radiation  fog  (thus, 
radiation  condition  1  is  not  met). 
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NOTE:  A  negative  result  from  any  of  these  rulepath  functions  causes  an  explanatory 
message  on  why  radiation  fog  is  not  possible. 


Figure  19. 
(visibi 1  it' 


Radiation  module 
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Similar  checks  are  done  using  dewpoint  depression  information, 
wet  ground,  and  boundary- 1 ayer  structure.  A  lot  of  time  has  been  spent  on 
representing  the  various  windspeeds  needed  for  fog  because  the  knowledge 
acquisition  phase  indicated  great  forecaster  concern  over  windspeed.  Therefore, 
in  addition  to  standard  checks  on  windspeed  limits  being  exceeded,  an  additional 
crosscheck  is  made  using  FOUS  data  to  ensure  proper  bound  ary- 1 ayer  and 
surface  windspeeds. 

The  radiation  fog  equation  is  then  executed.  The  result  of  the 
equation  is  a  duration  and  intensity  of  radiation  fog.  This  result  is,  in 
turn,  modified  usually  in  terms  of  the  duration,  by  the  meteorological  rules 
previously  "fired."  A  result  is  then  displayed  based  on  category  and 
time. 

If  the  controller  selects  the  path  of  visibility  currently  less 
than  3  mi  due  to  fog  or  haze,  then  the  job  of  the  radiation  module  is  to 
determine  improvement,  persistence,  or  further  degradation. 

Improvement  involves  the  breakout  routine  mentioned  under  the 
advection  module.  The  routine  is  basically  the  same  with  assessments  of 
breakout  being  made  based  on  PIREP,  satellite,  and  sounding  information  along 
with  a  surface  wind  assessment.  Persistence  is  based  in  part  on  low  surface 
visibility  at  surrounding  stations  indicating  the  extent  of  the  low  visibility 
problem  along  with  synoptic  information. 

Finally,  further  degradation  is  determined  by  integrating  synoptic 
information  and  by  assessments  such  as  moisture  on  windshields  (or  wet  ground 
checks)  indicating  sufficient  surface  layer  moisture  that  when  combined  with 
other  factors  could  indicate  decreasing  visibility.  Fig.  20  depicts  the 
radiation  module  in  the  visibility  less  than  a  3-mi  mode. 

Condition  2  radiation  fog  reflects  a  relatively  common  case  associated 
with  radiation  fogs  after  warm  frontal  passages,  and  is  based  on  our  interviews. 
This  condition  is  satisfied  if  a  warm  front  has  passed  north  of  the  station 
and  the  cold  frontal  passage  is  lagging.  The  synoptic  module,  in  combination 
with  the  radiation  module,  determines  whether  the  condition  exists.  After 
this,  the  radiation  module  proceeds  with  its  assessment  of  the  situation  and 
then  produces  a  specialized  message  if  all  radiation  condition  1  requirements 
are  satisfied. 

Finally,  a  third  condition  is  implied  throughout  the  radiation 
module.  If  condition  1  or  2  is  not  met,  then  messages  are  provided  to 
indicate  no  fog.  This  follows  the  Zeus  logic  of  first  assessing  advection 
fog,  then  radiation  fog;  if  neither  are  true,  then  no  fog  is  assumed  and 
associated  messages  are  printed. 


In  summary,  the  radiation  module  is  run  after  the  advection  module 
fails  to  identify  an  advection  fog  situation.  The  controller,  in  turn, 
determines  which  pathways  the  module  takes  in  terms  of  low  or  good  current 
visibility.  The  functions  of  the  module  are: 

•  Determine  radiation  fog  possibilities  by  independent  and 
synoptic  module  integration 

•  Determine  persistence,  improvement,  or  degradation  based  on 
the  breakout  routine  or  on  degradation-specific  radiation 
rules. 

The  early  use  of  the  conceptual  model  also  greatly  aided  development 
of  this  module. 


3.3.4  The  Synoptic  Module 

The  synoptic  module  is  really  the  heart  of  the  expert  system 
because  information  gathered  from  its  rules  are  fed  to  the  other  modules. 

The  objectives  of  the  module  are  twofold: 


•  Determine  location  of  synoptic  scale  systems 


•  Determine  movement  and  effect  of  the  movement  of 
synoptic  systems. 

Location  of  the  varying  synoptic  features  are  most  important.  For 
example,  a  high  near  TTN  (Trenton,  New  Jersey)  would  affect  weather  differently 
than  a  high  near  CLT  (Charlotte)  in  western  North  Carolina. 

Thus,  a  scheme  was  needed  to  geographically  identify  the  location 
of  the  various  weather  systems.  Several  methods  were  examined  to  depict  the 
location  of  systems.  One  obvious  method  of  using  real-time  information  with 
the  system  was  ruled  out  in  the  proof-of-concept  stage  because  it  was  felt 
that  such  an  effort  now  would  be  beyond  this  proof-of-concept  in  Al-meteorology. 


A  second  method  of  asking  the  user  for  latitude  and  longitude 
coordinates  was  deemed  to  be  too  rule- intensive  at  this  time.  Instead, 
through  our  terms  of  knowledge  acquisition  work,  we  found  that  forecasters 
tend  to  think  in  terms  of  regions.  Thus,  we  developed  a  map  (Fig.  21) 
divided  into  sections  for  positioning  of  systems. 

The  map  shows  six  areas  of  which  areas  2  and  3A  are  of  extreme 
importance. 


Figure  21.  Map  showing  sectors  used  by  syno 
to  identify  weather  system  location 
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It  appears  that  the  positioning  of  high-pressure  systems  (and  not 
so  much  low  pressure)  is  a  driving  force  behind  fog  formation.  For  example, 
highs  in  area  2  tend  to  influence  the  development  of  Atlantic  flow  and  highs 
in  region  3A  with  clear  skies  could  lead  to  the  radiational  case. 

For  high  pressure  systems,  three  characteristics  were  established: 

•  Movement  (north,  south,  east,  west,  stationary) 

•  Structure  (ridge,  center) 

•  Location. 

After  a  user  places  the  high  in  the  proper  position,  he/she  is 
asked  for  movement  and,  based  on  the  response,  various  actions  are  undertaken. 
These  actions  are  summarized  below  in  TABLE  13. 

TABLE  13.  Summary  of  High-Synoptic  Module  Results. 


High  Located  in 

Area. . .  Moving 


Action 


1 

Eastward 

Potential  for  advection- 
fog-1 ater  messages 

1 

Stationary,  north- 
south,  ridging 

No  fog  messages 

1 

Southeast 

Later  potential  for 
Radiation  fog  if 
moving  to  3A 

2 

Anywhere 

Generally  potential  sign 
of  advection  fog;  run 
advect ion  module  for 
further  assessment 

Ridging/stat ionary 

Same  as  above,  different 
messages 

3A 

E  astward 

Possible  advection  fog, 
run  advection  module  and 
radiation  module 

3A 

Stationary, 
ridging,  any 
other  direction 

Potential  for  radiation 
fog;  run  radiation  module 

38,  3C 

Anywhere 

Same  as  3A,  but  less 
emphasis  because  system 
is  farther  from  coast 

4 

Eastward 

Potential  for  radiation 
foq  later 

Anywhere  pls^ 

No  fog 
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The  high  functions  also  execute  under  the  persistent,  improvement, 
and  degradation  situations. 


For  example,  should  Atlantic  flow  be  established,  then  the  job  of 
the  synoptic  module  is  to  determine  whether  the  flow  can  maintain  itself  over 
the  next  6  to  12  h.  In  this  case,  movement  of  the  high  out  of  Area  2  (a 
cold-frontal  passage),  or  any  number  of  events  could  trigger  the  improvement 
forecast . 


areas . 
taken . 


Similarly,  low  pressure  systems  are  treated  according  to  the  same 
After  lows  are  placed  in  the  proper  area,  various  actions  are  under - 
These  are  surrmarized  in  TABLE  14  below. 


TABLE  14. 

Summary  of  Low-Synoptic 

Module  Results. 

Low  Located  in 
Area. . . 

Moving 

Action 

1 

Anywhere 

No  factor 

2 

Anywhere 

No  factor 

3A 

Anywhere 

Precipitation--can 't 
handle  messages 

3B 

Anywhere 

Precipitation--can 't 
handle  messages 

3C 

Anywhere 

Precipitation--can’t 
handle  messages 

4 

East 

Precipitation--can't 
handle  messages 

4 

Anywhere  else 

No  factor 

Lows  provide  an  interesting  case  resulting  in  one  of  two  actions: 
either  a  no-factor  result  or  precipitation  result.  The  no-factor  result 
means  just  that;  the  low  is  not  considered  by  the  advection  or  radiation 
module.  The  precipitation  results  are  a  cautionary  message  to  indicate  to 
the  users  that  Zeus  cannot  currently  handle  precipitation.  This  stems  from 
us  “drawing  the  line"  in  regards  to  the  proof-of-concept  and  not  going  into 
developing  rules  and  structures  tor  precipitation  forecasting.  It  was  felt 
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that  the  proof-of-concept  should  be  narrow  in  scope  and  that  an  outlet  should 
be  provided  to  later  expand  the  system  into  other  meteorological  areas.  The 
"can't  handle"  messages  provide  this  outlet. 

Cold  fronts  are  also  treated  in  terms  of  their  geographic  ’ocation 
and  movement.  Cold  fronts  are  geographically  determined  as  to  their  position 
relative  to  the  base  and  fall  into  the  categories  of: 

•  West  or  northwest  of  the  base  (common) 

•  South  or  southwest  of  the  base  (rare) 

•  North  or  northeast  of  the  base  (backdoor) 

•  East  (past  base) . 

Cold  fronts  are  broken  down  by  movement  into  the  categories  of: 

•  Will  pass  in  the  next  0  to  12  h 

t  Will  stall  within  100  mi 

•  Will  not  affect  (will  not  pass). 

Should  cold  fronts  pass  a  base  and  not  stall,  then  no  fog  will 
occur.  If  current  conditions  are  less  than  3  mi  in  fog  or  haze,  then  a 
cold  frontal  passage  (FROPA)  without  stalling  generates  an  automatic  clearing 
or  improvement  message.  If  the  cold  front  stalls  within  100  mi,  messages 
appear  alerting  the  forecaster  to  the  potential  of  a  wave  developing  on  the 
front --something  that  the  system  cannot  currently  handle.  Special  advisory 
messages  appear  on  the  handling  of  a  backdoor  cold  front  passage.  Finally, 
if  a  cold  front  is  east  of  or  will  not  affect  the  station,  then  the  cold 
front  is  deemed  to  be  of  no  factor. 

A  similar  structure  exists  for  warm  fronts.  Position  categories 

are : 

•  East  or  southeast  of  base  (coastal  front) 

•  South  or  southwest  of  base  (common) 

e  North  or  will  not  affect  base. 

Movement  categories  are: 

•  North  (or  pass  base) 

•  Stall. 


Warm  fronts  can  result  in  prolonged  periods  of  low  visibility  and 
precipitation  so  many  messages  appear  to  the  user  regarding  the  precipit at  ion 
possibi 1 ity--which  Zeus  cannot  handle  at  this  time.  Many  times,  however,  the 
warm  front  is  north  or  will  not  affect  the  station.  A  special  message  is 
transferred  to  the  radiation  module,  if  a  warm  front  has  just  passed  the 
base,  alerting  the  module  to  the  possibility  of  post-warm-frontal  radiation 
fog . 
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Messages  and  the  results  of  various  synoptic  rule  executions  are 
routinel"  passed  to  the  appropriate  module  first  as  a  block  of  mandatory 
information  such  as  synoptic  information  or  on  a  requested  basis  by  each 
individual  module.  The  passed  information,  in  most  cases,  relates  to  the 
changes  for  the  better,  worse,  or  no  change.  These  changes  are  represented 
by  setting  or  not  setting  one  of  the  three  change  conditions  (introduced 
earlier)  that,  although  not  grammatically  correct  in  our  language  representa¬ 
tion,  are  called: 

•  Change  good 

•  Change  bad 

•  No  change. 

As  an  example,  assuming  no  other  influential  factors  and  current 
visibility  greater  than  7  mi,  a  high  moving  eastward  from  area  1  would  trigger  a 
change-bad  condition.  This  is  because  a  high  moving  into  area  2  from  area  1 
could  eventually  result  in  Atlantic  flow  that,  in  turn,  could  lead  to  advection 
fog.  This  result  in  its  various  forms  would  be  transferred  to  the  advection 
module  for  further  processing.  Depending  on  the  execution  of  the  rules 
within  the  advection  module,  results  could  vary  from  providing  guidance  to 
the  forecast.  It  could  range  from  key  signs  to  examine  over  the  next 

6  to  18  h  to  actually  giving  6-h  visibility  category  advice. 

In  summary,  the  early  use  of  the  conceptual  model  provided  a 
framework  for  the  development  of  the  synoptic  module  that  is  really  the 
driving  force  behind  execution  of  any  of  the  advection  or  radiation  module 
rules . 

3.3.5  The  Concept  of  Time 

The  need  to  reason  with  and  use  time  has  been  a  recurring  problem 
not  only  in  AI ,  but  in  many  other  areas  of  computer  science.  In  meteorology, 
almost  all  numerical  models  (i.e.,  NGM,  LFM,  Spectral)  have  time  components 
and  time-related  differential  equations.  Meteorologists  deal  with  time 
systems  such  as  local  and  Greenwich  time  and  with  time  constraints  such 
as  the  TAF  deadl ine. 


The  easiest  way  to  create  a  time  subsystem  within  Zeus  is  to  divide 
the  24-h  day  into  daytime  and  nightime  periods  based  on  sunrise  and  sunset 
times.  Thus,  Zeus  has  built  in  sunrise  and  sunset  times  by  month,  rounded 
off  to  the  nearest  half-hour  period. 

The  day  and  night  division  allow  convenient  representation  of 
certain  rule  paths.  For  example,  fog  formation  times  were  derived  from  the 
executed  rules  within  the  modules  providing  time  increments.  These  meteoro¬ 
logical  increments  were  added  o  sunset  or  in  some  cases  subtracted  from 
sunrise  times  to  obtain  format  in  time. 
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In  the  breakout  routine  time  is  critical.  Breakout  time  is  derived 
similar  to  the  formation  time.  However,  the  rules  change  at  12  m.  because, 
based  on  climatology,  fog  lasting  into  afternoon  is  a  special  case.  Rules 
are  then  executed  to  determine  regional  extent  of  the  problem  and  various 
assessments  are  made.  Late  in  the  afternoon,  other  special  rules  are  used  to 
determine  reformation  of  the  fog  (degradation  from  an  already  existing 
situation)  near  sunset.  Thus,  sunset  time  becomes  extremely  important. 

Time  is  used  in  other  determinations  such  as  the  analysis  that  goes 
into  visibility  fog  actually  degrading  at  time  of  Zeus  run  and  occasionally 
within  the  synoptic  module. 

Finally,  given  the  entire  nature  of  the  visibility  forecasting 
problem,  we  have  overall  Zeus  system  time  constraints.  This  means  that  the 
system  is  useful  in  the  time  range  of  0  on,  out  to  18  h.  Most  advice  refers 
to  the  0-  to  1 2 - h  period.  For  forecasts  greater  than  18  h  in  advance,  more 
numerical  guidance  would  be  needed  as  Zeus  input.  We  decided  not  be  bring 
this  additional  information  into  Zeus  at  this  time  to  put  clear  time  constraints 
on  this  proof-of -concept  effort. 

The  Zeus  system  also  required  current  time  input .  Various  methods 
were  tried;  however,  we  decided  that  the  easiest  method  to  use  current  time 
within  the  system  was  to  create  a  batch  file  to  access  the  PC  system  clock. 
Current  time  is  passed  directly  to  Zeus  from  the  PC  clock  and  used  throughout 
the  program. 

Calendar  dates  are  also  received  from  the  PC  clock,  thereby 
providing  a  convenient  method  of  applying  monthly  climatological  rules. 

This  was  especially  useful  in  analyzing  the  various  base  wind  sectors  for 
Atlantic  flow  by  month. 

3.3.6  Expert  System  Internal  Rule  Structure 

The  rules  are  the  representation  of  the  knowledge  within  Zeus.  A 
rule  contains  one  or  more  IF  statements  followed  by  one  or  more  or  THEN  or 
ELSE  parts.  Notes  and  references  can  also  be  included  under  each  rule. 

The  rules  are  in  English  or  algebraic  expressions.  Rules  may  also 
contain  choices  in  the  THEN  part.  Choices  are  possible  major  goals  of  the 
system.  The  three  choices  that  Zeus  has  are  the  three  visibility  categories 
1  through  3.  Choices  can  include  a  probability  in  either  the  yes/no,  0  to  10, 
or  -100  to  -*-100  decision  systems.  In  our  case,  we  selected  the  0  to  10 
probabi 1 ity  system. 

The  rules  are  structured  along  the  lines  of  conditions.  A  condition 
is  a  statement  of  fact  or  potential  fact  Conditions  can  be  either  text  or 
mathemat i c al .  Text  can  be  true  or  false.  Each  condition  has  two  parts,  a 
qualifier  and  a  value.  Qualifiers  refer  to  the  part  of  the  condition  before 
the  verb.  The  values  are  possible  completion  phrases  for  the  rest  of  the 
cond  i  t  i on  . 
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[  Fig.  22  illustrates  the  rule  concept;  Fig.  23  illustrates  a  specific 

meteorological  rule. 


IF  conditions,  then  conditions  choices,  else  conditions  choices. 


Figure  22.  Typical  Zeus  rule  internal  structure. 


If  wind  is  360  to  040,  then  condition  1  is  met. 
Else  condition  1  is  not  met. 


Figure  23.  Example  of  rule  structure. 


The  condition  is  in  reality  the  entire  rule.  The  qualifers  are 
"wind  is"  and  "condition  1  is."  Values  are  "360  to  040,"  "met,"  "not  met." 

One  can  imagine  how  powerful  such  a  rule  structuring  system  can  be  in  terms  of 
improving  system  design.  We  were  able  to  isolate  qualifers  and  match  them 
with  existing  or  newly  created  values.  Thus,  many  times  when  faced  with  a 
new  rule  situation,  one  has  to  merely  search  the  existing  knowledge  base  to 
determine  whether  any  qualifers  exist. 

Upon  selection  for  inclusion  into  the  system,  a  split  screen 
appears  that  allows  for  piecing  together  of  rules.  Standard  IF,  THEN,  ELSE 
prompts  appear  and  the  user  is  guided  along  in  rule  development. 

The  IF  part  of  the  rule  is  a  set  of  conditions.  EXSYS,  the  software 
driver,  tests  the  conditions  against  input  to  see  if  the  IF  conditions  are 
true.  The  THEN  grouping  also  uses  conditions,  but  introduces  choices.  If 
the  "IF"  is  satisfactory,  then  the  "THEN"  is  executed  (otherwise,  the  ELSE  is 
executed ) . 


Each  rule  has  the  capability  to  draw  on  the  logic  structures 
operating  within  the  entire  system.  This  means  that  logic  operators  such  as 
NOT,  AND,  or  OR  can  be  used  almost  anywhere  in  system  development. 

Within  the  structure  of  Zeus,  two  major  facilities  exist  that  aid 
in  either  debugging  or  backtracking  distance.  The  facilities  are: 


•  Why 

t  Change  and  rerun. 


The  “why"  facility  allows  for  first,  second,  and  third  (inner)  tier 
reasoning.  By  indicating  the  "why"  facility,  the  user  can  readily  receive 
information  on  why  fog  was  advised  or  not  advised. 

The  change  and  rerun  command  allows  a  user  to  select  one  or  several 
parameters  for  "tweaking."  Change  and  reruns  usually  tap  into  the  various 
upfront  data  sheets.  A  forecaster  can,  therefore,  change  a  temperature  or 
the  position  of  a  synoptic  feature  to  understand  how  sensitive  the  forecast  is 
to  any  uncertainties  he/she  may  have  about  input  data. 


3.3.7  User  Interfaces  and  Result  Display 


The  use  of  user  interfaces  throughout  AI  has  been  a  topic  of  great 
discussion  at  many  recent  computer  conferences.  User  interfaces  range  from 
windows  with  mouse-controlled  functions  to  detailed  diagrams  with  light  pen 
pointer  functions. 


We  decided  in  this  proof-of -concept  effort  to  limit  ourselves  to 
simplistic  user  interfaces  and  spend  most  of  our  effort  in  the  structuring 
and  knowledge  acquisition  aspects  of  the  study.  We  adopted  this  philosophy 
for  one  primary  reason:  the  current  proof-of -concept  nature  of  AI -meteorology. 
We  believe  that  once  representation  structures  and  acquisition  techniques 
have  been  established,  then  sophisticated  user  interfaces  can  be  developed. 

This  philosophy  is  analagous  to  what  has  transpired  in  the  medical  AI  field 
where  considerable  recent  effort  has  been  spent  in  real-time  data  acquisition, 
display,  and  user  interfaces  only  after  representation  and  acquisition  schemes 
were  wel 1  developed. 


We  initially  identified,  by  the  conceptual  models,  that  real-time 
data  was  required  by  the  system.  We  decided  not  to  pursue  automatic  data 
acquisition  and  instead  concentrated  on  manual  input  that  can  be  readily 
transferred  to  automatic  input  at  a  later  date. 


A  basic  program  was  created  to  handle  the  input.  This  program  runs 
just  prior  to  the  main  body  of  Zeus  and  feeds  information  directly  to  the 
expert  system. 

The  first  type  of  input  that  is  passed  is  called  a  variable  (V). 

The  variable  has  a  unique  identification  number  assigned  to  it  for  tracking 
purposes.  The  value  of  the  variable  can  be  either  integer  or  real. 


A  second  type  of  input  transfer  involves  the  use  of  qualifiers.  A 
qualifer  is  typically  completed  by  a  linking  verb  such  as: 

The  month  i s : 

January 

February 

March 


etc . 


“The  month  is"  is  the  qualifer  ("is"  is  the  linking  verb),  and  January, 
February,  etc.,  are  the  conditions  of  the  qualifer.  Qualifers  such  as  "The 
month  is  September"  are  routinely  passed  to  Zeus  from  the  input  program. 

Key  weather  observing  stations  contained  in  the  input  for 
Seymour  Johnson  are:  ILM,  EWN,  HAT,  2DP,  ECG,  ORF,  and  Seymour  Johnson 
itself. 


Key  weather  observing  stations  contained  in  the  input  for  Dover 
are:  BWI,  SBY,  WAL,  ACY,  ILG,  and  Dover  itself. 

Key  weather  observing  stations  contained  in  the  input  for  Simmons 
are:  GSB,  NKT,  EWN,  ILM,  MYR,  and  Simmons  itself.  Should  observations  not 
be  available,  latest  or  nearest  observations  are  substituted.  The  required 
inputs  are  temperature  and  dewpoint  for  the  regional  stations  and  basic 
meteorological  input  for  the  base. 

Base  inputs  include  FOUS  (PBL)  data,  closest  sounding  winds,  sunset 
temperature  parameters,  sky  cover,  and  current  visibility.  Time  and  month 
are  automatically  filled  in  by  the  PC  clock  batch  access  program. 

Figs.  24  to  45  illustrate  the  user  input  requirements  for  base 
and  regional  stations. 

Seymour  Johnson's  input  screen  is  similar  to  Fort  Bragg's  screen 
except  for  the  page  two-station  page.  Dover's  input  screen  contains  several 
different  parameters  than  the  other  bases  such  as  fraction  of  cloud  cover  at 
sunset,  sea  surface  temperature,  and  FOUS  6-h  relative  humidity  (boundary 
layer).  These  factors  are  used  more  often  at  Dover  than  at  the  other 
bases . 


Figs.  24  to  45  also  include  the  four  windows  and  map  used  to 
input  synoptic  information.  The  windows  are  cursor  controlled  for  display 
purposes.  The  map  provides  reference  to  the  areas  and  includes  a  distance 
l  bull seye  reference . 

I 
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**  INPUTS  FOR  ZEUS  METEOROLOGICAL  ADVISORY  SYSTEM  ( GSB )  ** 

<P AGE  1  of  3 > 


CURRENT 

DATE 

( mm- 

-dd-yy )  : 

[06-14-861 

CURRENT  TIME 

(  0001 

-2400 ) 

[16581 

SUNSET 

SURFACE  TEMP . 

(F)  : 

[75  1 

SUNSET  DEW- 

POINT 

TEMP 

.  (F) 

[65 

1 

WIND 

DIRECTION 

(deg)  : 

(  060  1 

WIND  SPEED 

( mph ) 

[  5 

1 

HAT 

850MB  WIND  DIR 

(deg )  : 

[  0501 

HAT  850MB  WIND  SPD 

( mph ) 

[  20 

1 

FOUS 

06 

HRS  WIND 

DIR 

(deg ) : 

[  030  J 

FOUS  06  HRS 

WIND 

SPD 

( mph  ) 

[  12 

1 

FOUS 

12 

HRS  WIND 

DIR 

(deg) : 

C  0  40  ) 

FOUS  12  HRS 

WIND 

SPD 

( mph ) 

[15 

1 

FOUS 

18 

HRS  WIND 

DIR 

(deg ) : 

[0501 

FOUS  18  HRS 

WIND 

SPD 

(mph) 

[10 

1 

FOUS 

24 

HRS  WIND 

DIR 

(deg ) : 

[  070  1 

FOUS  24  HRS 

WIND 

SPD 

(mph ) 

[  10 

1 

SKY 

( See 

below) : 

[3  1 

VISIBILITY 

( miles  ) 

l  7 

1 

(SKY  Codes:  1=CLR,  2=SCT,  3=BKN/  4=OVC,  5  =  -X,  6  =  X) 


U3e  arrow  keys  to  move  cursor  to  where  you  want  it,  then  just  type  input. 
When  done,  hit  FI  key  to  see  the  next  page.  There  are  three  input  pages. 
<f>  <  4->  <-»•><«->  <Home>  <End>  <Ctrl-»>  <Ctr  l«->=Move  Cursor,  Fl=Next  Page,  F10  =  Exit 


**  INPUTS  FOR  ZEUS  METEOROLOGICAL  ADVISORY  SYSTEM  (GSB)  ** 

< PAGE  2  of  3 > 


GSB  2DP  ORF  EWN  ECG  ILM 


SURFACE  TEMP.  (F) : 
DEW-POINT  TEMP.  (F) : 


(75  ]  [79  ]  (75  ] 
[64  ]  [69  ]  (63  ] 


(80  1  [78  ]  [83  ] 
[59  ]  (62  ]  [66  ] 


Get  the  latest  surface  observations  for  the  listed  stations  and  enter  them 
Use  arrow  keys  to  move  cursor.  When  done,  hit  FI  key  to  see  the  next  page 
<  t>  <<->  <Home  >  <End>  <Ctr  l-»  <Ctr  l<->  =Move  Cursor,  Fl  =  Next  Page,  F10  =  Exit 


Figure  25.  Inputs  for  Zeus  meteorological  advisory  system  (GSB). 


High  Pressure  System  is  located  in 


Fl=Next  Page,  F10=Exi t  (GSB) 

ut,  GSB. 


Fl=Next  Page,  F18=Exit  (GSB) 


Warn  Front  is  located 

1,  east  or  south  of  station  (coastal  front) 


Arrows:OX*>:Move  Cursor,  <Enter>=Flip  Window,  Fl=Next  Page,  F10=Exi t  (GSB) 

Figure  29.  Nap-related  input,  GSB. 


Cold  Front 
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Exit  <GS6) 


**  INPUTS  FOR  ZEUS  METEOROLOGICAL  ADVISORY  SYSTEM  (FBG) 


? 

■v 


k. 


*  * 


<PAGE  1  of  3  > 


CURRENT 

DATE 

( mm- 

-dd-yy)  : 

(06-24-861 

CURRENT  TIME 

(  0001- 

2400) 

[1705] 

SUNSET 

SURFACE  TEMP. 

(F)  : 

(75  1 

SUNSET  DEW- 

POINT 

TEMP  . 

(F) 

[68 

I 

WIND 

DIRECTION 

(deg) : 

(240  1 

WIND  SPEED 

(mph ) 

[  4 

1 

HAT 

850MB  WIND  DIR 

(deg ) : 

(  324  ) 

HAT  850MB  WIND  SPD 

(mph ) 

[15 

1 

FOUS 

06 

HRS  WIND 

DIR 

(deg) : 

[  200  ] 

FOUS  06  HRS 

WIND 

SPD 

( mph ) 

[  5 

1 

FOUS 

12 

HRS  WIND 

DIR 

(deg ) : 

[  190  ] 

FOUS  12  HRS 

WIND 

SPD 

(mph) 

(8 

) 

FOUS 

18 

HR S  WIND 

DIR 

(deg) : 

I  200] 

FOUS  18  HRS 

WIND 

SPD 

(mph ) 

[10 

] 

FOUS 

24 

HRS  WIND 

DIR 

(deg ) : 

[2101 

FOUS  24  HRS 

WIND 

SPD 

(mph) 

[10 

1 

SKY 

( See 

below) : 

11  ] 

VISIBILITY 

(mi les ) 

[7 

] 

(SKY  codes:  1*CLR,  2=SCT,  3=BKN,  4=OVC,  5=-X,  6=X) 


Use  arrow  keys  to  move  cursor  to  where  you  want  It,  then  just  type  input 
When  done,  hit  FI  key  to  see  the  next  page.  There  are  three  input  pages. 
<‘f>  <4'>  <-*>«->  <Home> <End> <Ctrl-»> <Ctr  l«->=Move  Cursor,  Fl=Next  Page,  FI0  =  Exit 


Figure  31.  Inputs  for  Zeus  meteorological  advisory  system  (FBG) 
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**  INPUTS  FOR  ZEUS  METEOROLOGICAL  ADVISORY  SYSTEM  ( FBG )  ** 

<P AGE  2  of  3 > 


FBG 

GSB 

NKT 

EWN 

ILM 

MYR 

SURFACE  TEMP. 
DEW-POINT  TEMP. 

(F)  : 
(F)  : 

(74  ] 
(68  ] 

(73  ] 
(65  ] 

(65  ] 
(62  ] 

(67  J 
(65  ] 

(66  ] 
(62  J 

(71  J 
(65  ] 

Get  the  latest  surface  observations  for  the  listed  stations  and  enter  them 
Use  arrow  keys  to  move  cursor.  When  done,  hit  FI  key  to  see  the  next  page 
<t><i><-*-><-*-><HomeXEnd><Ctr  !-►>  <Ctr  k- >  =  Move  Cursor,  Fl  =  Next  Page,  F10  =  Exit 


Figure  32.  Inputs  for  Zeus  meteorological  advisory  system  (FBG). 


<-*><<->:Hove  Cursor 


Exit  (FBG) 


Arrows :0>0->:Move  Cursor,  <Enter>=Flip  Window,  Fl=Next  Page,  F10:Exit  (F6G) 

Figure  36.  Map-related  input,  FBG. 


Cold  Front  is  located 


related 


ZEUS  A  KNOWLEDGE-BASED  EXPERT  SV5TEH  THAT  ASSISTS  IN  2/2 ^ 
PREDICTING  VISIBILI  <U>  GEOHET  TECHNOLOGIES  INC 
GERMANTOWN  MD  M  J  STUNDER  ET  AL  15  JAN  87 
GEOMET-EAF-1725  AFGL-TR-87-0819  F/G  12/5  NL 


**  INPUTS  FOR  ZEUS  METEOROLOGICAL  ADVISORY  SYSTEM  (DOV) 


*  * 


<P AGE  1  of  4 > 


CURRENT  DATE 

(mm- 

-dd-yy) : 

[06-14-861 

CURRENT  TIME 

(0001- 

2400  )  : 

[1750] 

SUNSET  SURFACE  TEMP. 

(F)  : 

[77  1 

SUNSET  DEW-POINT 

TEMP. 

(F)  : 

(68 

1 

WIND 

DIRECTION 

(deg)  : 

[260) 

WIND  SPEED 

( mph ) : 

[12 

1 

ACY  1 

950MB  WIND  DIR 

(deg) : 

[  252  J 

ACY  850MB  WIND  SPD 

( mph ) : 

132 

] 

FOUS 

06  HRS  WIND 

DIR 

(deg) : 

[2201 

FOUS  06  HRS 

WIND 

SPD 

(mph) : 

[10 

] 

FO  US 

12  HRS  WIND 

DIR 

( deg ) : 

1  205) 

FOUS  12  HRS 

WIND 

SPD 

( mph ) : 

I  5 

] 

FOUS 

18  HRS  WIND 

DIR 

(deg) : 

[1901 

FOUS  18  HRS 

WIND 

SPD 

(mph) : 

[5 

] 

FOUS 

24  HRS  WIND 

DIR 

(deg) : 

[180] 

FOUS  24  HRS 

WIND 

SPD 

( mph ) : 

[5 

1 

SKY 

( See 

below) : 

[2  1 

VISIBILITY 

(miles ) : 

[7 

1 

(SKY  Codes:  1=CLR,  2=SCT,  3=BKN,  4=OVC,  5=-X,  6=X) 


Use  arrow  keys  to  move  cursor  to  where  you  want  it,  then  just  type  input. 
When  done,  hit  FI  key  to  see  the  next  page.  There  are  four  input  pages. 
<^><^><-»<4-><Home><EndXCtr l-»<Ctrl«->=Move  Cursor,  Fl=Next  Page,  F10=Exit 


**  INPUTS  FOR  ZEUS  METEOROLOGICAL  ADVISORY  SYSTEM  (DOV)  ** 


**  INPUTS  FOR  ZEUS  METEOROLOGICAL  ADVISORY  SYSTEM  (DOV)  ** 


<PAGE  3  of  4> 


DOV 

SBY 

ILG 

WAL 

ACY 

BWI 

SURFACE  TEMP. 
DEW-POINT  TEMP. 

(F)  : 
(F)  : 

[80  ] 
[65  ] 

[78  ] 
[64  1 

[82  ] 
[64  ] 

[79  ] 
[68  ] 

[75  1 
[69  ] 

[80  ] 
[63  ] 

Get  the  latest  surface  observations  for  the  listed  stations  and  enter  them. 
Use  arrow  keys  to  move  cursor.  When  done,  hit  FI  key  to  see  the  next  page. 
<f><^><-*><«-><Home><End><Ctrl-*XCtrl«*>=Move  Cursor,  Fl=Next  Page,  F10=Exit 


Figure  40.  Inputs  for  Zeus  meteorological  advisory  system  (DOV). 
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Arrow :<-»<+>:Nove  Cursor,  <Enter>=Flip  Uindow,  Fl=Next  Page,  FlfcExit  (DOU) 

Figure  41.  Map-related  input,  DOV. 


High  Press 


Fi=Next  Page,  FlfcExit  (DOV) 

ut,  DQV. 


l!dr«  Front 


Fl=Next  Page,  F18=Exi t  (DOW) 


High  Press  Low  Press  Haw  Front 
Cold  Front  is  located 


Fl=Next  Page,  FlfcExit  (DOV) 


TABLE  15  indicates  where  Zeus  input  program  variables/qualifiers 
are  used  within  the  main  Zeus  program. 

The  user  interface  idea  was  modeled  along  the  lines  of  the  forecaster 
worksheet  that  appears  in  Fig.  46.  Base  forecasters  are  required  to  fill  out 
the  forecaster  worksheet  in  his/her  TAF  preparation  procedure.  These  sheets 
provide  for  a  convenient  recording  of  meteorological  information  for  ready 
base  access. 

Finally,  the  output  display  of  Zeus  results  consisted  of  advisory 
text  and  visibility  categories.  Sample  output  appears  in  Figs.  47  and  48. 
Unfortunately,  it  is  difficult  to  graphically  depict  fog  on  a  computer  screen 
(short  of  turning  the  screen  white  when  fog  is  expected)!  We  did  consider  a 
graph  of  visibility  over  time,  but  we  finally  decided  on  explanatory  text 
that  fit  nicely  into  the  segmented  advisory  messages  being  produced  by  each 
module  used  to  input  synoptic  information. 

In  the  next  section,  we  describe  the  various  knowledge  acquisition 
procedures  used  to  "fill  out"  or  "flesh  out"  the  structure  presented  in 
this  section. 


TABLE  15.  Zeus  Input  Program  Versus  Zeus  Main  Program  Use 

(Example:  Simmons  AAF). 


Input  Variable/ 

Qual if ier 

Source 

Where  Used  (in  Zeus) 

Time 

System  clock 

Many  rules. 

Month 

System  clock 

Sunrise/sunset  rules;  wind 
direction-Atlantic  flow  rules; 
any  climatology-related 
rules. 

Wind  direction 
and  speed 

Simmons 

surface  observation 

Radiative  and  advective 
modules.  Speed  used  in 
breakup  (faster  means  breakup 
quicker)  and  in  formation 
(i.e.,  high  winds  indicate  no 
fog). 

HAT  850  mbar,  wind 
direction  and 
speed 

Hatteras  sounding 

Atlantic  flow  determination; 
speed  important  for  radiative 
fog. 

FOUS 

Wilmington  (for 
Simmons) 

Combined  with  other  conditions- 
determines  whether  Atlantic 
flow  could  exist  or  whether 
other  flow  regimes  exist. 

Sky,  visibility 

Simmons 

surface  observation 

Sky:  combined  with  other 
conditions  determines  radiative 
potential . 

Visibility:  important 
variable,  used  in  major 
triggering  rules:  if  low, 
then  system  follows  path 
of  persistent  low  visibility 
or  improvement;  if  greater 
than  3,  system  determines 
whether  low  visibility  will 
exist  in  the  future. 

Surface  temperature 
and  dewpoint  at 
surrounding  stations 

Surrounding  station 
observations 

Used  in  temperature  minus 
dewpoint  spread  analysis 
of  eastward  stations  if 
speed  analysis  is  less 
than  10*. 
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DATE 


I.  SYNOPTIC  SITUATION 


FORECAST  WORKSHEET/VERIFICATION  LOG 


RADAR  INDICATES _ 

SATELLITE  INDICATES _ 

LAWC/SXUS  AT  _ Z  INDICATES 


ARE  BOTH  OP/DOWN  YES/NO 


AFGWC  MWA _ CURRENT  TEMP-DEWPOINT _ 

II.  FORECAST  CONSIDERATIONS: 

A.  FOUS  62  KWBC 

BOUNDARY  WINDS  R1  R2  R3  SI  PRECIP  PRESS 

6  HR _ Z  _ _ _  _ 

I2HR _ Z  _ _ _  _ 

1BHR _ Z  _ _ _  _ 

24HR _ Z  _ _ _  _ 

B.  FJUE  S3  KGWC 

700  MB  HT  UP/DOWN  500  MB  HT  UP/DOWN  ARE  BOTH  UP/DOWN  YES/NO  SI _ 

C.  FOUE  05  KWBC 

TSTH  WINDS  C/V 

6  HR _ Z  _  _  _ 

12KR _ Z  _  _  _ 

18HR _ C  _  _  _ 

24HR _ Z  _  _  _ 

D.  LFH-.  IS  THE  LFM  INITIALIZED  PROPERLY?  YES/NO 

VORTICITY  ADVECTION  (POS/NEG/NEUT)  CURR: _  12HR: _  24HR: _ 

THICKNESS  ADVECTION  (INC/DEC/NEUT)  CURR: _  12HR: _  24HR: _ 

GSB  IN/OUT  700  MB  704  ISOPLETH  CURR: _  12HR: _  24HR: _ 

5400  M  LOCATION  (FALL/WINTER  ONLY)  CURR: _  12HR: _  24HR: _ 

850  MB  TEMP  ADVECTION  (W/C/NEUT)  CURR: _  12HR: _  24HR: _ 

E.  TURBO:  FANH  2  INDICATES  TURBO  YES/NO,  LEVELS _ TO  _ 

SKEW-T  INDICATES  TURBO  f  SPEED /DIRECTION  SHEAR  PRESENT)  YES/NO, LEVELS  _ 

_  FANA  INDICATES  TURBC  BELOW  lO.OOOFEET  YES /NO 

F.  ICG:  FANH  2  INDICATES  ICG  YES/NO,  TYPE _ LEVELS  _ TO  _ 

FREEZING  LEVEL  _ FT  AT  _ .  IS  SIG  MOISTURE  AVAILABLE  AT  THE 

FREEZING  LEVEL?  YES/NO  FANA  INDICATES  ICG  BELOW  10,000  FEET  YES/NO 

G.  RDU  FCST  _  _ 


CURR: _ 

_  12HR: _ 

_  24HR: 

CURR: 

_  12HR: _ 

_  24HR: 

CURR: 

_  12HR: _ 

_  24HR: 

CURR: 

12HR: 

_  24HR: 

CURR: 

12HR: 

24HR: 

H.  ARE  AREA  OR  TERMINAL  METWATCH  ADVISORIES  IN  FTSC!  FOR  ICG,  TURBC,  TSTMS,  ETC? 


-  Page  1  of  2  pages 

U  03 

OEC  84  ..  .. 

(Continued) 


Figure  46.  Seynour  Johnson  forecaster's  worksheet. 
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INITIAL  FORECAST  OESSEMINATICN :  T/A  _ Z  t-GMEDS  _ Z  ATAU  Z 

NOTE:  APPEND  A  HARD  COPY  OF  THE  CSB  TAF  TO  THIS  FORM,  CHECK  AND  INITIAL  FOR  ACCURACY. 

AMD  #  _  BEFORE  AFTER  REASON?  — 

CSB  TAF  AMD  _ 


DISSEMINATION  T/A  _ Z  COMEDS  _ Z 

FORECASTER  REMARKS,  NOTES,  HINDSIGHT,  SECOND  THOUGHTS,  MINI  REVIEWS!!!!!!!!!! 


TIME  (L) 

TOfP 

PA 

VERIFICATION: 
FCST  HR 
FCST  C/V 
OBSVD  c r 


PERSISTENCE 


P«g«  2  of  2  pages 

Figure  46.  Seymour  Johnson  forecaster's  worksheet  (Concluded). 


,4  WaVA*  *.* 


1 


Advise  Visibility  Category  3 


70 

Current  time  (in  hhmm)  =  1900.000000 

Current  visibility  (in  miles)  =  7.000000 

ADVECTIVE  FOG  FORMATION  TIME  (APPROXIMATELY)  =  0.000000 

High  is  moving  eastward  or  northeastward  or  north  from  New  England 

area.  Monitor  dew-points  and  any  gradual  windshifts  over  next  several 

hours  for  any  change  in  temperature  minus  dewpoint  differences  in 

surrounding  area;  look  for  any  increase  in  easterly  component  to  the 

wind  at  the  surface  and  aloft  in  boundary  layer. 

Even  though  synoptic  features  indicate  surface  Atlantic  flow  needed  foi 
ADVECTIVE  FOG  is  present,  necessary  boundary  layer  features  are  NOT 
NOV  present. 

Overall  synoptic  situation  is  not  favorable  for  RADIATIVE  FOG  formatioi 
during  the  next  12  hours. 

RADIATION  FOG  EXPECTED  FORMATION  TIME  (APPROXIMATELY)  =  0.000000 


M- 


1  Advise  Visibility  Category  1 
85 

Current  time  (in  hhmrn)  =  1900.000000 

Current  visibility  (in  miles)  *  3.700000 

ADVECTIVE  FOG  FORMATION  TIME  (APPROXIMATELY)  =  0.000000 

High  is  either  becoming  stationary  or  is  ridging  down  coast  which 

increases  nighttime  RADIATIVE  FOG  and  daytime  HAZE  possibilities  if 

summer,  or  nighttime  RADIATIVE  FOG  possibilities  if  winter. 

RADIATION  FOG  EXPECTED  FORMATION  TIME  (APPROXIMATELY)  =  300.000000 
Backdoor  coldfront  is  to  your  north.  As  a  reminder,  make  sure  front  is 
not  moving  down  coast  as  this  could  affect  forecasts  dramatically. 

Check  surface  obs .  for  little  frontal  movement;  check  850mb  flow  for 
signs  of  weakening; Make  sure  winds  are  parallel  to  front .. .anything 
else... then  be  suspicious  of  movement  southward.... 

RADIATIVE  FOG  is  expected  to  form  around  radiative  fog  formation  time 
and  then  lower  to  below  1  mile.  Keep  an  eye  out  for  any  intervening 
clouds,  but  conditions  look  favorable  at  this  time. 

Current  date  =  07-04-80 


Figure  48.  Typical  Zeus  output. 
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Section  4.0 
KNOWLEDGE  ACQUISITION 


"Knowledge"  itself  is  the  key  ingredient  to  any  expert  system 
development.  Thus,  knowledge  acquisition  is  probably  the  most  important 
aspect  of  developing  an  expert  system.  It  is  important,  not  only  because 
knowledge  is  necessary  to  make  the  expert  system  run,  but  it  is  also  important 
because  of  the  importance  of  expert  system  developer-user  interaction.  The 
term  "knowledge  engineer"  has  been  associated  with  the  person  who  collects 
the  knowledge  and  formulates  the  presentation  schemes. 


The  knowledge  engineer  usually  has  intensive  interactions  with  the 
domain  experts.  This  poses  some  interesting  situations  in  many  AI  applications 
where  Al-oriented  knowledge  engineers  try  to  become  pseudo-experts  in  a 
particular  field.  Sometimes  it  is  successful  as  in  several  medical  AI 
systems,  where  after  a  year  or  two,  the  knowledge  engineers  become  pseudo¬ 
doctors  or  medical  technicians.  In  other  expert  system  developments,  it  has 
not  been  very  successful. 


In  the  meteorology  field,  there  is  no  reason  why  meteorologists 
cannot  become  knowledge  engineers  themselves  provided  they  have  the  proper 
training.  A  readily  apparent  analogy  can  be  drawn  between  computer  scientists 
and  meteorologists.  Most  meteorologists  can  program  in  FORTRAN  or  other 
common  computer  languages.  They  also  know  how  to  logically  design  a  structured 
program.  Consequently,  they  can  do  much  of  the  work  themselves.  Should  a 
more  theoretical  programming  problem  be  encountered,  a  computer  scientist 
could  be  called  in  to  help  the  meteorologist  resolve  the  difficulty.  Similarly, 
in  AI -meteorology,  meteorologists  can  be  trained  as  knowledge  engineers  and 
conduct  interviews  and  structure  expert  system  themselves.  Should  a  major 
difficulty  be  encountered,  the  meteorologist  could  call  on  a  (pure)  knowledge 
engineer  or  theorist.  This  is  the  approach  that  has  been  adopted  by  DuPont 
where  over  300  successful  KBES  projects  have  been  developed  by  non-AI-oriented 
personnel  to  date. 
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The  use  of  trained  knowledge  engineers  with  meteorological  backgrounds 
provides  another  great  advantage  in  terms  of  time,  because  the  meteorological 
knowledge  engineer  is  already  familar  with  the  domain  and  could  save  considerable 
amounts  of  effort  in  interacting  with  fellow  (domain)  meteorologists. 


4.1  GENERAL  KNOWLEDGE  ACQUISITION  MECHANISMS 

The  process  of  acquiring  knowledge  can  be  divided  into  four  categories: 

1.  Literature  review 

2.  Structured  interview 

3.  Study  of  domain  experts  performance  of  tasks 

4.  Performance  of  experts  on  tough  cases. 

The  literature  review  usually  occurs  first  and  sets  the  stage  for 
the  later  interviews.  The  review  is  straightforward  with  domain  and  other 
resources  examined. 

The  structured  interview  results  from  a  close  literature  review  and 
combining  that  knowledge  with  any  similar  knowledge  engineering  experience  to 
form  a  complete  picture  of  the  situation.  This  "first-pass"  or  conceptual 
model  is  then  used  as  a  source  of  questions  to  the  experts.  The  interview 
leads  to  additions  and  deletions  of  information. 

The  third  category  Involves  studying  the  actions  of  the  experts 
while  they  are  engaged  in  typical  tasks.  The  object  of  the  specific  study  is 
to  look  for  commonalities  in  terms  of  goals,  data  records  produced,  imagery 
viewed,  or  information  the  experts  like  to  have  available. 

The  final  category  involves  the  way  the  expert  handles  a  tough 
case.  The  expert  could  be  presented  a  previous  problem  and  be  asked  to  solve 
it.  The  knowledge  engineer  then  looks  for  subtle  or  refined  aspects  of 
the  expert's  reasoning  and  uses  this  Information  to  further  enhance  the 
knowledge  data  base. 

4.2  APPROACH  TO  ZEUS  KNOWLEDGE  ACQUISITION 

4.2.1  Compilation  of  Rules  and  Techniques  from  Literature  Review 

The  literature  search  consisted  of  examining  the  following  resources 
for  useful  information: 


•  GEOMET's  corporate  library 

•  NOAA  library  (Rockville,  Maryland) 


•  North  Carolina  State  University  library 
(Raleigh,  North  Carolina) 

•  AWS  library  (Scott  Air  Force  Base,  Illinois) 

•  DTIC/NTIS. 

Twenty-two  visibility  documents  were  borrowed  on  interlibrary 
loan  from  the  AWS  library  along  with  numerous  copied  "bits  and  pieces"  of 
other  key  documents  such  as  RUSSWO's.  Several  key  documents  were  copied 
from  the  NOAA  library  and  a  cross-check  was  made  to  avoid  duplication  of 
documents  from  other  sources.  A  list  of  reference  material  appears  in 
Appendix  B. 

GEOMET  requested  DTIC  meteorological  search  terms  are  listed  in 
TABLE  16.  The  search  was  requested  for  the  years  1950-present.  A  quick  scan 
of  the  search  indicates  many  RUSSWO  document  citations  for  other  airbases, 
which  will  not  serve  our  purpose. 

In  addition,  GEOMET  was  placed  on  the  DTIC  recurring  reports 
list  for  the  created  visibility  search  strategy  and  received  biweekly  DTIC 
updates  on  new  visibility  developments.  This  greatly  aided  in  any  modification 
or  additions  to  the  knowledge  base  due  to  new  research  or  reports  over  the 
course  of  the  project. 

A  typical  visibility  document  evaluation  consists  of  the  following 
questions  asked  internally  by  a  GEOMET  meteorologist: 

•  Is  the  document  specific  to  the  mid-Atlantic,  Southeast 
region,  or  study  airbase?  If  the  answer  is  yes, 
special  attention  is  given  to  any  rules  mentioned 
within  the  case  review  text  or  special  meteorological 
analysis  procedures  presented.  For  example,  special 
note  has  been  made  of  the  satellite  "burnoff  from  the 
edges  technique"  used  to  forecast  fog  dissipation 
described  in  a  Seymour  Oohnson/SE  United  States 
document . 

•  Ooes  the  document  contain  a  general  review  of  visibility 
techniques?  If  this  is  the  case,  particular  attention 
Is  paid  to  subsections  regarding  specific  types  of 

fog  (advection,  radiation,  etc.)  or  phenomena  (drizzle, 
stratus,  etc.). 

e  Is  the  document  more  equation/model  oriented? 

Several  evaluated  documents  derive  numerical  models 
for  predicting  fog  (see,  for  example,  reference  11). 
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TABLE  16.  GEOMET  Requested  DTIC  Search  Terms  for  Low- 
Visibility  References. 


First-Level  Search  Terms 

Slant  Range 

Visibility 

Visual  Flight 

Visual  Range 

Second-Level  Search  Terms 

Anticyclones 

Ice  Fog 

Atmospheric  Condensation 

Jet  Streams 

Atmospheric  Motion 

Lapse  Rate 

Atmospheric  Precipitation 

Lightning 

Atmospheric  Refraction 

Meteorological  Phenomena 

Atmospheric  Temperature 

Meteorology 

Barometric  Pressure 

Microbarometric  Wave 

Ceiling 

Monsoons 

Cloud  Cover 

Nimbostratus  Clouds 

Clouds 

Rain 

Cold  Fog 

Sea  Breeze 

Cold  Fronts 

Snow 

Crosswinds 

Snow  Cover 

Cumulonimbus  Clouds 

Snowdrifts 

Cumulus  Clouds 

Snowfields 

Cyclones 

Storms 

Dew 

Stratus  Clouds 

Dust  Storms 

Temperature  Inversion 

Fog 

Thunderstorms 

Fronts  (Meteorology) 

Tornadoes 

Geostrophic  Wind 

Tropical  Cyclones 

Gusts 

Wind 

Hail 

Haze 

Hydrometeors 

Rules  were  recorded  on  4"  X  6"  cards  for  easy  reference. 


Other  information  such  as  simplistic  equations  are  also  recorded  on 
the  4"  x  6"  cards.  These  simplistic  equations  were  thought  to  be  useful  in 
deriving  future  rule  required  quantities  such  as  mixing  ratios,  boundary  layer 
depth,  etc. 

4.2.2  Compilation  of  Rules  from  Airbase-Specific  Literature  and  Airbase 
Personnel 


Interviews  with  airbase  personnel  were  conducted  from  February  24 
through  28,  1986.  These  interactions  consisted  of  the  following  four 
components: 

•  GEOMET  briefing  to  AWS  personnel  on  the  AI/KBES 
project . 

t  General  round-table  discussion  with  available  AWS 

personnel  on  visibility  forecasting  at  the  particular 
airbase. 

•  Individual  interviews  with  available  AWS  personnel. 

•  Identification  of  important  base-specific  meteorology 
and  documents. 


A  listing  of  AWS  and  civilian  personnel  interviewed  appears  in  TABLE  17. 


GEOMET  personnel  briefed  all  available  AWS  personnel  on  the  nature 
of  the  project.  These  briefings  typically  took  an  hour  and  included  many 
exchanges  of  ideas  and  information.  GEOMET  meteorologists  began  the  discussion 
by  questioning  the  audience  on  whether  they  had  ever  heard  of  AI  and  then  by 
giving  an  overview  of  AI  and  expert  systems. 

Careful  emphasis  was  given  to  the  fact  that  this  effort  is  both 
exploratory  { i . e . ,  feasibility  study,  proof-of-concept  ideas)  and  not  meant 
to  replace  humans.  A  particular  effort  was  made  to  stress  that  the  KBES  is 
really  a  knowledge  or  meteorological  advi sory  system.  GEOMET  personnel  have 
been  sensitive  even  prior  to  contract  start  to  the  potential  public  perception 
that  AI  is  a  "2001"  technology  and  that  it  does  away  with  humans.  It  is 
our  opinion  that  the  bases  reacted  favorably  to  the  project  description,  are 
not  perceiving  AI  in  the  wrong  manner,  and  are  more  than  willing  to  cooperate 
throughout  the  project  as  evidenced  by  the  amount  of  information  obtained  and 
enthusiasm  shown  during  the  interviews. 


A  second  component  of  the  visits  involved  a  round-table  discussion 
of  conditions  that  can  cause  low  visibility  at  the  various  airbases.  This 
discussion  laid  the  groundwork  for  the  interviews. 


Interviewed  later. 


The  individual  interviews  initially  consisted  of  discussing  the 
experience  of  the  AWS  person.  This  included  questions  such  as  educational 
background,  forecasting/observer  background,  general  meteorological  back¬ 
ground,  and  length  of  time  at  the  base.  This  preliminary  discussion  will  aid 
us  in  determining  the  quality  of  information  obtained  f-om  the  interviewee. 
Obviously,  the  AWS  person  who  has  been  at  the  station  3  years  and  in  AWS 
11  years  has  more  experience  than  a  person  who  has  been  at  the  station 
3  months  and  who  has  only  8  weeks  of  formal  meteorological  training.  Yet, 
the  less  experienced  forecaster  may  have  valuable  insights  into  either  a 
low-visibility  case  he/ she  was  specifically  involved  with  or  in  placement  and 
use  of  the  KBES. 

The  next  step  in  the  interview  was  to  quickly  establish  the 
generalized  meteorological  situations  under  which  certain  parameters  or 
events  were  likely  to  occur.  For  example  (based  on  the  initial  discussion), 
North  Carolina  low-visibility  situations  may  be  divided  into  wind  flow 
regimes  such  as: 

•  Atlantic  flow 

•  Southwesterly  flow. 

Questions  were  directed  toward  key  parameters  that  can  be  used  to 
predict  the  onset  of  one  of  the  flow- associated  weather  conditions.  AWS 
personnel  were  given  several  "what  if"  conditions  and  asked  how  they  would 
respond  as  a  forecaster.  We  present  in  Appendix  C  the  conversation  held 
between  Mark  Stunder  (GEOMET  meteorologist)  and  Bob  Madison  (Fort  Bragg 
civilian  meteorologist)  as  an  example  of  a  typical  GEOMET-AWS  meteorological 
interview.  Most  interviews  were  recorded  on  cassette  tapes. 

Simplistic  fog  routines  were  obtained  from  Dover  and  Pope  AFB 
(located  20  mi  from  Simmons).  These  routines  are  shown  in  Figs.  50, 

51,  and  52.  The  parameters  of  these  routines  were  included  in  the  various 
rulepaths  as  indicated  by  the  various  interviews. 

Additional  interviews  were  held  by  conference  telephone  calls  or 
correspondence.  All  information  was  encoded  into  rule  format  for  use  in 
EXSYS. 
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forecast  chktust 

STRATUS/FOG 


1.  data  REQUIRED!  date _ 

*«  POB  aaxbsun  temperature _ op 

b.  POB  surface  wind  direction  (1500E) _ _ 

c.  POB  surface  tesrperature  (1500E)_ _ op 

d.  POB  dew-point  temperature  (1500E)  op 

e.  PCS  dew-point  depression  (l  500 E)  °? 

f.  Vfaa  there  stratus/fog  this  momlngT  Yes _ No _ • 

£•  Vas,  or  '.ill  there  be  local  showers(l200L-0000E)T  Yes _ No _ 

b.  Is  POB  1500E  dew-point  65°F  or  higher?  Yes _ No _ 

1.  Is  or  will  the  wind  flow  ecae  inland  fron  overvater?  Yes  No _ 

2.  STEPSt 

a*  If  la  is  less  than  55°Ft  the  aethod  is  not  applicable,  Stop, 

b,  Ehter  the  forecast  disgran  with  1b  and  1e.  Yes _ No _ Stop, 

c«  If  the  point  falls  in  the  southwest  quadrant  "possible*  area  and 

at  least  two  of  If,  1g  and  1h  are  yes,  forecast  Yes _ , 

otherwise  forecast  No _ . 

d.  If  the  point  falls  in  the  Northwest  "possible"  area,  and  1i  is 
yes,  forecast  les _  otherwise  forecast  No _ , 


3.  VSHmCATTCSi 

a.  Forecast:  Tes _ 

b.  Tine  of  occurence_ 

c.  Lowest  Condition^ 

d.  Remarks  1  _ 


Observed:  Yes  No_ 

_  Tine  of  Dissipation, 


appendix  2 

TFHM 

Det  U,  5  v Wg,  Fope  AFB 


Source:  Clark  (1965) 


r»  y 


Figure  50.  Forecast  checkl ist--stratus/fog,  Pope  AFB. 
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1 500E  Surface  Wind  Direction  (in  degrees)  Versus  1500E  Dew-Point 
Depression  (in  degrees  Fahrenheit)* 

Figure  51.  Stratus/fog  forecast  chart  (source:  reference  21). 


I, 


FOG  STUDY  WORKSHEET 


1  . 


High  pressure  centered  between  38n  and  45N  east 
of  70w. 


2. 


Dewpoint  equal  to  or  higher  than  sea  temp(ARQ  Sxus 
8  KNYC  and  KORF) . 


3. 


Subsidence  and/or  nocturnal  inversion  from  500  ft  to 
1500  ft. 


Surface  winds  less  than  8  kts  with  a  040  to  180 
direction. 


5. 

6. 


Clear  or  scattered  clouds  at  sunset. 


FOUS  61  KWBC  data  (use  PHL)  shows  high  RH  (greater 
than  75%)  in  boundary  layer  and  wind  direction  in 
NE  to  SE  quadrant  with  speed  less  than  15  kts. 


Area  of  moisture  over  new  Jersey  and  Delmarva  at 
8  50mbs  (outline  dewpoint  depressions  of  less  than 
5. 


Important  No  Fog  parameters 

a.)  radiation  fog  -  850mb  ridge  to  pass  over 
DOV  during  night. 


b.)  advection  fog  -  moderate  to  strong  low  to 
move  into  Ohio  Valley  during  period. 


Figure  52.  Dover  fog  checklist. 
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Section  5.0 
EVALUATION 

5.1  EVALUATION  OF  EXPERT  SYSTEMS 


Technically  speaking,  the  development  of  an  expert  system  is  always 
being  evaluated  because  the  development  under  each  phase  is  considering 
questions  such  as: 

e  Is  the  knowledge  representation  adequate  or  should  it 
be  modified? 

o  Can  users  easily  interact  with  the  system? 

•  Are  rules  consistent  with  the  expert's  opinion? 

Evaluation  of  expert  systems  can  be  classified  into  two  broad 
categories:  quantitative  and  qualitative.  Quantitative  evaluation  includes 
derivation  of  statistical  results  from  either  real-time  or  past  events.  This 
involves  compilation  of  observed  versus  expert  system  predicted  results. 
Qualitative  analyses  is  much  more  difficult,  but  centers  around  the  funda¬ 
mental  question:  "Did  the  expert  system  help  the  user?"  If  the  expert  system 
is  able  to  meet  the  needs  of  the  user  and  provide  him  with  advice  on  recommenda 
tions,  then  the  system  has  fulfilled  its  job. 

5.2  ZEUS  RULE  EVALUATION 


Our  evaluation  philosophy  involved  both  the  generic  quantitative 
and  qualitative  options  described  above.  Individual  rules  or  rulepaths  were 
evaluated  based  on  a  combination  of  how  they  were  used  by  the  experts  in 
reaching  a  decision  and  on  how  they  were  related  to  the  physical  reasoning. 
Individual  rules  were  also  examined  using  several  past  cases  and  by  comparing 
the  rules  used  by  experts  at  one  base  with  rules  used  by  experts  at  another 
base. 
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5.3  BASE  FORECASTER  SURVEY  FORMS 
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5.3.1  Philosophy/Methodology 


Standardized  forms  were  developed  for  each  base  to  reflect  the 
qualitative  aspect  of  expert  system  evaluation.  It  was  important  to  have 
this  record  of  information  to  see: 


When  Zeus  was  used  in  TAF  preparation 
How  Zeus  was  used  in  TAF  preparation 
What  Zeus  problems  arise. 


The  forms  also  served  an  additional  purpose  of  requiring  the  users  to  run 
Zeus  over  a  several -month  period. 


The  forms  follow  our  philosophy  of  allowing  the  user  to  truly 
dictate  how  he  or  she  will  use  the  system.  This  effort  has  been  one  of  the 
first  AI  efforts  that  allowed  users  early  hands-on  experience  with  an  expert 
system  still  in  the  proof-of-concept  phase.  The  initial  format  of  the  form 
appears  in  Fig.  53. 


This  form  is  structured  along  general  lines  and  was  used  early  to 
get  initial  opinions  of  the  system. 


After  the  first  set  of  evaluation  forms  were  received  at  GEOMET, 
and  based  on  discussions  with  the  AFGl  Project  Officer  and  base  personnel, 
modifications  to  the  evaluation  form  were  made.  The  revised  form  (Fig.  54) 
was  organized  to  capture  forecaster  experience  in  using  the  system  during  TAF 
preparation  as  well  as  other  times. 


5.3.2  Statistical  Summary 


Survey  forms  were  left  with  each  of  the  three  bases  when  Zeus  was 
demonstrated.  Forecasters  were  asked  to  fill  out  a  survey  form  each  time 
they  used  the  system.  A  total  of  143  survey  forms  were  completed  and  returned 
to  GEOMET,  including  45  using  the  original  format  and  98  using  the  revised 
format . 


A  tabulation  of  the  answers  received  on  45  original  forms  is  shown 
in  TABLE  18.  The  answers  received  on  98  revised  forms  are  tabulated  in 
TABLE  19. 


It  is  clear  that  Zeus  was  well  received  by  the  forecasters  that 
used  it.  Of  the  45  that  responded  on  the  first  form,  42  liked  the  system;  of 
the  98  that  responded  on  the  second  form,  76  liked  the  system.  The  first 
form  focused  on  introductory  responses  to  the  system.  The  system  was  under¬ 
stood  by  41  of  45  users;  however,  15  of  45  users  indicated  they  would  prefer 
a  less  burdensome  method  of  entering  data.  The  explanatory  features  of 
systems  were  found  adequate  by  40  of  45  users,  and  32  of  45  thought  they 
would  be  helpful  on  a  daily  routine  basis. 


-114- 


,■>>>; 


2.  Time 


(Local) 


3.  Did  you  like  the  way  the  system  interacted  with  you?  (Circle  one) 
Yes  No 

Why?  _ 


4.  Did  you  understand  all  the  questions  that  it  asked  you? 
Yes  No 

What  questions  did  you  not  understand?  (if  any) 


5.  Would  you  prefer  a  different  method  of  entering  the  information  that 
the  system  requested? 

Yes  No 

Why?  _ 


6.  Did  you  use  the  system  ...  (Check  one) 

as  needed  _ 

just  prior  to  TAF  time  (1  hour)  _ 

other  times  (specify)  _ 

7.  Are  the  explanation  features  adequate? 

Yes  No 

If  no,  why  not?  _ 

8.  Do  you  think  this  system  will  be  helpful  in  your  everyday  routine? 
(Circle  one) 

Yes  No 

Why?  _ 


Figure  53.  Initial  user  evaluation  form. 
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Date 


Time  _  (Local) 

Name _ 

1.  Did  this  system  assist  you  in  preparing  the  new  TAF?  (Circle  one) 
Yes  No  Not  applicable 


2.  Did  you  like  the  way  the  system  interacted  with  you  during  this 
session?  (Circle  one) 

Yes  No 

Comments: 


3.  Did  this  particular  run  of  the  system  cause  you  to:  (circle  one  or 
morel 

a.  Amend  the  current  TAF? 

b.  Think  about  a  weather  factor  that  you  might  have  missed 

otherwise?  Which  factor?  _ 

c.  Confirm  the  current  TAF  as  still  being  OK? 

d.  Have  no  reaction? 

e.  Other  (please  explain)  _ 


Comments: 


4.  Any  other  remarks? 


Figure  54.  Revised  user  evaluation  form. 
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TABLE  18.  Summary  of  Forecaster  Replies  on  Original  Form  (see  Fig.  53). 


Question  Number 
(See  Form) 

Answer 

Dover  AFB 

Seymour 
Johnson  AFB 

Fort 

Bragg  AAF 

Total 

3 

Yes 

6 

22 

14 

42 

No 

0 

2 

1 

3 

4 

Yes 

6 

21 

14 

41 

No 

0 

3 

1 

4 

5 

Yes 

1 

9 

5 

15 

No 

5 

15 

10 

30 

6 

As  needed 

0 

10 

5 

15 

TAF 

5 

10 

9 

24 

Other 

1 

4 

1 

6 

7 

Yes 

6 

21 

13 

40 

No 

0 

3 

2 

5 

8 

Yes 

2 

19 

11 

32 

No 

4 

5 

4 

13 

Number  of  Forms 

6 

24 

15 

45 

TABLE  19.  Summary  of  Forecaster  Replies  on  Revised  Form  (see  Fig.  54) 


Question  Number 

(See  Form)  Answer  Dover  AFB 

Seymour 
Johnson  AFB 

Fort 

Bragg  AAF 

Total 

1 

Yes 

9 

30 

0 

39 

No 

13 

32 

11 

56 

Not  Applicable 

1 

1 

1 

3 

2 

Yes 

17 

50 

9 

76 

No 

6 

11 

3 

20 

No  Response 

2 

2 

3 

Amend 

1 

1 

0 

2 

WX  Factor 

0 

2 

0 

2 

Confirm 

11 

38 

1 

50 

No  Reaction 

7 

17 

10 

34 

Other 

3 

3 

1 

7 

No  Response 

1 

2 

0 

3 

4 

Other 

Number  of  Forms 
(Added  to  Table) 


23  +  6 
29 


24  +  63 
87 


12  +15  98  +  45 

27  143 


The  Zeus  system  was  most  commonly  used  when  the  TAP  was  being 
prepared  as  indicated  by  24  of  45  uses  on  the  old  form  and  39  of  98  uses  on 
the  new  form.  In  addition,  responses  on  the  new  form  indicated  that  52  of 
98  uses  led  to  confirming  or  amending  the  TAF.  This  suggests  that  even  if 
Zeus  was  not  used  in  preparing  the  TAF,  it  was  subsequently  used  to  review 
the  need  to  amend  the  TAF.  It  is  interesting  that  in  two  cases,  forecasters 
actually  decided  to  amend  their  forecasts  on  the  basis  of  assistance  from 
Zeus,  and  in  two  other  cases,  forecasters  were  alerted  to  weather  factors 
that  they  might  not  otherwise  have  considered. 


Overall,  the  forecasters  were  very  receptive  to  the  computerized 
expert  system  approach  to  providing  them  with  advice.  The  user  liked  the 
system  in  118  out  of  143  reported  uses,  or  about  82  percent  of  the  time.  The 
use  is  not  limited  to  official  forecast  preparation  (TAF),  because  only 
63  out  of  143  uses  or  44  percent  were  reported  as  assisting  in  TAF  preparation. 

5.3.3  User  Comments 

The  survey  forms  provided  a  method  for  users  to  indicate  problems 
with  Zeus.  This  is  one  of  the  major  benefits  that  was  anticipated  from  the 
distribution  of  Zeus  to  active  forecasters.  All  three  bases  reported  that 
the  structure  and  general  knowledge  content  of  Zeus  seem  to  be  fine;  however, 
the  following  five  problems  were  commented  on: 


Problem 


Location 


1.  Sunshine  question 

2.  HAT  850  mbar  wind  direction/FOUS 

3.  Synoptic  descriptors 

4.  Air  trajectory  question 

5 .  Speed 


All  bases 
Seymour  Johnson 
All  bases 
Fort  Bragg 
Dover 


Problem  1:  Sunshine  Question 


The  sunshine  question  appearing  below  has  caused  the  greatest 
problem  among  the  bases.  The  question  asked  by  Zeus  is: 

Number  of  hours  of  sunshine  today?  (Note:  Answer  1  if 
time  now  is  between  sunrise  and  1200  noon;  take  a  good 
guess  if  current  time  is  between  1200  noon  and  sunset;  and 
finally,  try  to  really  get  a  good  estimte  from  the  day's 
observations  if  the  current  time  is  after  sunset  and 
before  sunrise) 

This  sunshine  parameter  was  determined  to  be  necessary  based  on 
conversations  with  several  base  personnel.  The  physical  reasoning  was  that 
an  air  mass  under  cloudy  skies  all  day  and  advected  into  the  base  in  question 
has  a  greater  moisture  supply  than  if  skies  are  clear.  Sufficient  cooling 
should  cause  the  temperature-dewpoint  depression  to  narrow  and  fog  to  form. 


Too  much  sunshine,  for  example,  during  the  day  may  dry  out  the  air  and 
restrict  fog  formation  at  night.  The  rules-oriented  programming  using 
results  of  this  question  requires  an  answer  (thus,  the  "1"  answer  for  the 
morning  period).  Bases  are  having  difficulty  in  determining  amount  of 
sunshine  that  has  either  been  forecasted  or  has  occurred. 

We  can  see  two  solutions  to  the  problem: 

1.  Change  question  to  read:  "Enter  number  of  clear, 
scattered,  broken,  or  overcast  hours"  with  Zeus, 
determining  sunshine  hours  from  these  results.  The 
scattered,  broken,  or  overcast  hours  can  be  compiled 
from  the  WBAN  observation  forms  at  the  station. 

2.  Eliminate  the  question.  One  alternative  is  to  use  a 
substitute  input  variable  of  cloud  cover  at  sunset 
(this  is  in  use  at  Dover  only)  and  incorporate  it  at 
Seymour  Johnson  and  Fort  Bragg.  (In  reality,  Dover 
has  both  the  standard  sunshine  question  and  clouds  at 
sunset  input  requirement  because  personnel  tend  to 
look  at  both.) 

Problem  2:  HAT  850  mbar  Wind  Direction/FQUS  Restrictive 

An  interesting  result  regarding  boundary  layer  winds/FOUS  has  come 
out  of  the  evaluation  at  Seymour  Johnson.  Initial  knowledge-engineering- 
related  discussions  (February  1986)  with  Seymour  Johnson  personnel  indicated 
the  use  of  boundary  layer  (<  3000  ft)  winds  at  Raleigh  as  one  potential 
trajectory-related  trigger  that  could  lead  to  Atlantic  flow  (and  advective 
fog).  Clear  limits  were  placed  on  the  FOUS  boundary  wind  direction  and  on 
HAT  directions.  However,  evaluations  have  indicated  that  the  limits  are  too 
restrictive.  For  example,  a  change  from  040*  to  039*  ( i . e . ,  across  a 
HAT  850-mb-wind-direction  limit)  will  trigger  a  big  change  in  Zeus  output 
(low  visibility  to  good  visibility).  In  selecting  these  limits,  the  initial 
hope  was  to  usefully  classify  the  boundary  layer  wind  direction,  but  it 
appears  that  the  system  Is  too  sensitive  to  this  one  parameter.  One  reason 
may  be  that  the  FOUS  6-h  boundary  layer  output  is  not  a  good  representation 
of  the  actual  back  trajectory. 

Another  reason,  which  has  been  suggested  by  Seymour  Johnson  personnel, 
is  the  850-mbar  surface  is  too  high  to  represent  the  boundary  layer  flow. 

They  have  suggested  that  the  chart  F4531-Surface/geostrophic  wind  and  relative 
vorticity  be  examined  to  determine  the  direction  of  the  boundary  layer  flow. 

This  may  provide  a  more  realistically  oriented  estimate  than  just  one  sounding 
station.  There  is  a  need  to  incorporate  this  product  into  Zeus.  Many 
Seymour  Johnson  forecasters  are  now  examining  this  chart  to  determine 
whether  an  easterly  flow  or  "driftw  exists. 


Problem  3:  Synoptic  Description 

All  bases  have  reported  some  problems  in  visualizing  where  synoptic 
systems  are  located.  Many  personnel  have  suggested  a  fixed-map  that  will 
clearly  indicate  the  boundary  regions  of  the  various  quadrants  that  are  used 
in  Zeus. 

Seymour  Johnson  forecasters  have  indicated  difficulty  in  determining 
which  response  to  answer  when  faced  with  a  ridge  line  down  the  east  coast. 

In  response  to  comments  we  have  added  a  ridge  line  question  into  the  map 
software  used  with  Zeus. 

One  potential  difficulty  that  may  not  be  resolved  under  this  effort 
is  to  come  up  with  a  scheme  that  will  differentiate  the  dominance  of  one 
system  over  another.  For  example,  given  a  high  over  New  England  and  a  cold 
front  over  the  Ohio  Valley  (and  approaching),  which  one  will  dominate? 

Currently,  the  system  asks  the  user  whether  the  front  will  pass  and  makes 
judgments  (using  other  parameters)  if  the  user  answers  no. 

The  system  also  cannot  handle  restrictions  in  visibility  caused  by 
precipitation  such  as  is  found  with  overrunning  conditions,  low-pressure 
passages,  thunderstorms,  or  unusual  weather  events  (we  had  to  draw  the  line 
somewhere).  However,  messages  are  flashed  on  the  screen  to  indicate  that  the 
system  is  not  able  to  handle  the  current  situation.  Some  general  meteorological 
advice  is  provided  to  the  user  regarding  the  situation. 

Problem  4:  Air  Trajectory  Question 

At  Fort  Bragg,  a  problem  was  identified  with  the  question  regarding 
air  trajectories  under  cloudy  or  clear  skies.  This  question  was  established 
to  get  a  better  sense  of  moisture  content  along  the  trajectory.  Fort  Bragg 
personnel  have  indicated  that  this  question  can  be  removed  without  major 
meteorological  consequences.  There  is  a  need  for  more  extensive  checking  of 
past  results  with  and  without  the  question  for  the  consequences  before 
removing  this  question  from  the  logic. 

Problem  5:  Speed 

Prompts  on  the  monitor  to  indicate  that  Zeus  is  working  have  been 
installed.  Dover  forecasters  found  that  it  takes  approximately  0.5  h  to  run 
the  system;  other  forecasters  have  reported  taking  approximately  10  min. 

Discussions  with  the  Detco  at  Dover  revealed  the  following  two 
problems  that  have  been  causing  run  delays: 

1.  Physical  system  location 

2.  Run  again  prompt. 


Moving  the  system  to  the  counter  area  of  the  station  where  maps  and 
charts  are  readily  available  Improved  the  total  runtime  to  10-15  min. 


Additionally,  at  the  end  of  a  Zeus  session,  the  "Y"  key  was  hit  in  response 
to  the  "Run  Again"  question,  and  the  system  left  on.  If  the  "N"  key  is  hit, 
the  system  must  reread  all  rules  before  the  next  execution.  This  was  a 
considerable  time  saver  in  terms  of  start-up  processing  time  and  alleviated 
the  speed  concerns. 

5.4  STATISTICAL  EVALUATION 

Two  types  of  forecast  accuracy  comparisons  were  made.  One  compari¬ 
son  was  based  on  forecast  advice  from  the  Zeus  system  during  interactive 
sessions  with  Zeus  by  forecasters  while  the  system  was  available  at  the  three 
airbases.  The  results  of  this  evaluation  are  discussed  in  Section  5.4.1. 

The  other  type  of  comparison  was  based  on  running  Zeus  on  historical  data  and 
comparing  the  forecasts  with  observations.  This  required  a  mechanical  and 
objective  method  of  interaction  with  Zeus  so  that  the  user  could  not  influence 
the  system  forecast.  This  comparison  is  discussed  in  Section  5.4.2. 

5.4.1  Zeus  User  Forecasts 

The  Zeus  system  was  run  during  the  time  the  forecaster  was  preparing 
the  TAF  (four  times  a  day)  or  a  revision  to  the  TAF.  The  forecaster  would 
obtain  the  input  data  requested  by  Zeus  from  the  latest  observations  from  the 
teletype  circuits,  local  observations,  facsimile  charts,  or  other  information 
available  at  the  time. 

The  forecast  and  observed  visibility  category  for  validation 
purposes  was  the  lowest  visibility  category  over  the  next  12  h.  No  attempt 
was  made  to  validate  the  specific  timing  of  low  visibility  events. 

During  the  period  that  the  Zeus  system  was  available  at  the  three 
bases,  forecasters  were  requested  to  print  the  results  of  Zeus  sessions  and 
to  send  them  to  GEOMET  along  with  a  survey  form.  We  received  printed  results  of 
36  sessions  from  Dover  AFB,  36  sessions  from  Fort  Bragg,  and  104  sessions  from 
Seymour  Johnson  AFB.  When  cases  were  eliminated  for  which  a  visibility 
forecast  was  not  applicable  or  not  made  by  Zeus  because  of  precipitation 
influences,  there  were  29  cases  for  Dover,  29  cases  for  Fort  Bragg  and 
100  cases  for  Seymour  Johnson.  TABLES  20  through  22  show  three-by-three 
contingency  tables  for  these  cases,  using  the  three  operational  visibility 
categories  that  are  applicable  to  each  base.  The  tables  show  the  numbers  of 
occurrences  of  each  forecast  category  in  terms  of  what  category  was  actually 
observed. 


I 

! 


TABLE  20.  Contingencies  of  Zeus  Forecasts  and 
Observations  of  Three  Categories  of  Visibility  at  Dover  AFB. 


Observed  Visibility 

3 

2 

1 

Forecast 

Visibility 

>  2  mi 

>  1/2  and  <  2  mi 

<1/2  mi 

3 

20 

2 

3 

2 

1 

0 

0 

1 

1 

1 

1 

TABLE 

21 

Contingencies  of  Zeus  Forecasts 

and 

Observations 

of 

Three  Categories  of  Visibility  at  Fort  Bragg 

Observed  Visibility 

Forecast 

3 

2 

1 

>  2  mi 

>  3/4  and  <  2  mi 

<  3/4  mi 

Visibility 

3 

25 

3 

1 

2 

0 

0 

0 

1 

0 

0 

0 

TABLE  22.  Contingencies  of  Zeus  Forecasts  and 
Observations  of  Three  Categories  of  Visibility  at  Seymour  Johnson  AFB. 

Observed  Visibility 


3  _ 2  _ 1_ 

Forecast 

Visibility  >  3  mi  >1  and  <  3  mi  <  1  mi 


TABLE  23  shows  a  contingency  table  for  results  at  all  three  bases 
combined.  We  have  used  these  results  to  compute  skill  scores  (SS)  and 
P-scores  (PS)  by  means  of  the  following  relationships: 


R  -  E 


T  -  E 


where 


E.i 

T 


expected  number  of  correct  forecasts 
number  of  correct  forecasts 
total  number  of  forecasts 
number  of  forecasts  in  category  1 
number  of  observations  in  category  1. 


The  above  relationship  was  presented  by  Panofsky  and  Brier  (1968) 


In  this  relationship  for  the  skill  score,  the  expected  number  of 
correct  forecasts  (E)  is  based  on  chance.  If  there  is  no  relationship 
between  the  forecast  and  the  observation,  this  is  the  number  of  correct 
forecasts  that  can  be  expected. 


The  P- score  can  be  represented  by: 


PS  =  - 
T 


3  3 

2  Z  Cij 
1=1  j=l 
jfH 


where 


Cij  =  number  of  forecasts  in  category  i  when  category  j  was 
observed. 


TABLE  23.  Contingencies  of  Zeus  Forecasts  and 
Observations  of  Three  Categories  of  Visibi 1 1ty--Combined  Results. 


Forecast 

Visibility 

Observed  Visibility 

3 

2 

1 

3 

124 

8 

10 

2 

3 

4 

0 

1 

3 

2 

4 

-124- 


rr, 


Equation  (3)  is  derived  from  the  following  relationship  for  P-score 
(Panofsky  and  Brier  1968). 

ps  -  1  r  N 

^  2  2  (fmn  ■  Emn)^  (*) 

"  m=l  n=l 

where 

N  =  number  of  cases 

r  =  number  of  classes  for  the  event  being  forecast 
fjk  =  forecast  probability  that  the  event  occurs 

E  =  1  or  0  according  to  whether  event  occurs  or  not. 

r 

Z  fmn  =  1  (5) 

m=l 

In  the  case  being  evaluated,  we  have  divided  the  cases  into  nine  classes  as 
represented  by  the  contingency  table.  One  of  these  nine  classes  was  forecast 
to  occur.  That  class  is  assigned  fmn  =  1;  all  the  other  classes  are  assigned 
fmn  =  0.  Actually,  there  are  only  three  forecast  categories,  but  each  forecast 
category  can  be  subdivided  into  three  subcategories,  according  to  which  of 
the  three  observed  visibility  classes  was  subsequently  observed.  If  the 
classes  are  numbered  from  one  to  nine,  starting  with  upper-left  category  of 
the  contingency  table  and  proceeding  across  the  first,  then  the  second,  and 
finally  the  third  row,  for  each  case  that  class  1,  5,  or  9  occurs. 

(fmn  '  Emn)^  =  (1  -  1)2  -  0. 

That  is,  the  event  will  have  been  forecast  and  will  have  occurred. 

For  each  case  with  any  other  class,  there  will  be 

(fmn  '  Emn)^  =  (1  -0)2=1 
and  the  other  will  be 

(fmn  ■  Emn)^  =  ( 0  -  1 ) 2  =  0 . 

For  all  cases  with  a  class  other  than  1,  5,  or  9,  a  value  of  two  must 
be  summed.  Therefore,  we  can  write  our  summation  over  all  N  cases  as  indicated 
in  Equation  (3).  That  is,  two  is  summed  C i j  times,  where  cij  is.  the  class 
designated  by  forecast  category  i  and  observed  category  j,  except  when  i  =  j, 
we  sum  the  value  zero. 

The  skill  score  gives  the  fraction  of  cases  that  are  correctly 
forecast  from  among  the  number  of  cases  that  occur  beyond  the  number  that  are 
expected  to  be  correctly  forecast  by  chance.  If  the  number  of  correct 
forecasts  is  equal  to  or  less  than  the  randomly  expected  number,  then  a  zero 
or  negative  score  occurs.  If  50  percent  of  the  cases  above  the  number  of 
expected  correct  forecasts  are  correctly  forecast,  a  score  of  0.5  occurs. 
Obviously,  if  all  forecasts  are  correct,  a  perfect  score  of  1.0  occurs. 


The  P-score  is  a  method  of  evaluating  forecasts,  especially 
probability  forecasts,  that  gives  minimum  weight  to  a  high  probability 
estimate  for  a  correct  forecast  and  maximum  weight  to  high  probability 
estimate  for  an  incorrect  forecast.  Because  the  forecasts  here  are  all 
defined  to  be  certain  forecasts  with  an  assigned  probability  of  1,  the 
forecaster  gets  a  score  of  0  when  correct  and  2  when  wrong.  As  a  result,  the 
P-score  is  simply  twice  the  fraction  of  incorrect  forecasts.  It  is  provided 
primarily  as  a  convenience  in  comparing  the  results  from  this  study  with 
results  from  other  studies  that  provide  P-scores.  A  score  of  0.28  indicates 
that  14  percent  of  the  forecasts  are  incorrect. 

TABLE  24  shows  the  skill  scores  and  the  P-scores  calculated  for 
each  base  and  for  the  results  of  all  bases  combined  using  the  data  presented 
in  TABLES  20  to  23.  The  best  skill  score  of  0.46  was  obtained  for  Seymour 
Johnson.  The  best  P-score  of  0.28  was  obtained  both  for  Seymour  Johnson  and 
Fort  Bragg. 


TABLE  24.  Skill  Scores  and  P-Scores  for  Zeus  Forecasts. 


Base 

Skill  Score 

P-Score 

Dover  AFB 

0.16 

0.55 

Fort  Bragg 

0.00 

0.28 

Seymour  Johnson  AFB 

0.46 

0.28 

Combined  Result 

0.35 

0.33 

The  combined  results  for  all  three  bases  gave  a  skill  score  of  0.35 
and  a  P-score  of  0.33.  The  results  for  Dover  and  Fort  Bragg  are  less  reliabl 
than  the  results  for  Seymour  Johnson  due  to  the  much  smaller  number  of  cases. 


To  provide  more  insight  into  what  value  can  be  anticipated  from  the 
Zeus  advice,  forecasts  made  by  forecasters  at  Seymour  Johnson  AFB  (the 
published  TAF)  and  for  a  nearby  National  Weather  Service  station  (Raleigh, 
North  Carolina)  were  obtained  for  the  same  periods  that  the  Zeus  forecasts 
were  made.  The  three-by-three  contingency  tables  for  these  forecasts  are 
shown  in  TABLES  25  and  26.  The  data  shown  included  the  same  100  time  periods 
for  the  Seymour  Johnson  forecasters;  however,  only  83  of  the  100  forecasts 
were  obtained  for  the  Raleigh  forecasts. 


TABLE  25.  Contingencies  of  Seymour  Johnson  AFB  Forecaster 
Forecasts  and  Observations  of  Three  Categories  of  Visibility. 


Observed  Visibility 


Forecast 

Visibility  3  2  1 


3 

2 

1 


73 

7 

0 


4 

3 

2 


9 

1 

1 


TABLE  26.  Contingencies  of  Raleigh,  North  Carolina,  Forecaster 
Forecasts  and  Observations  of  Three  Categories  of  Visibility. 


Observed  Visibility 


Forecast 

Visibility 


3 


2 


1 


Skill  scores  and  P-scores  for  the  forecasters  at  Seymour  Johnson 
and  at  Raleigh  are  listed  in  TABLE  27.  The  skill  scores  at  both  locations 
are  about  equal,  but  significantly  less  than  was  obtained  for  Zeus,  i.e.,  0. 
for  Zeus  compared  to  0.23  for  Seymour  Johnson  forecasters  and  0.24  for 
Raleigh  forecasters.  The  P-scores  were  0.46  for  Seymour  Johnson  forecasters 
and  0.75  for  Raleigh  forecasters.  Both  values  are  significantly  higher  and 
thus,  poorer  than  the  0.28  obtained  for  Zeus. 

TABLE  27.  Skill  Scores  and  P-Scores  for  Seymour  Johnson 
and  Raleigh,  North  Carolina,  Forecasters. 


Skill  Score 


P-Score 


Seymour  Johnson  AFB 
Raleigh,  North  Carolina 


As  a  means  of  comparison,  a  skill  score  was  computed  to  compare  the 
improvement  in  P-scores  that  is  obtained  by  the  Zeus  system  over  the  fore¬ 
casters  above  without  Zeus.  This  was  computed  as  follows: 

Pf 

where 


Pf  =  P-score  of  forecaster 
P i  =  P-score  of  Zeus. 

The  skill  score  for  Zeus  in  comparison  to  Seymour  Johnson  forecasters  is 
0.39.  In  comparison  to  Raleigh  forecasters  the  Zeus  skill  score  is  0.63. 

5.4.2  Historical  Evaluation 

To  get  a  more  seasonally  balanced  evaluation  of  Zeus  than  was  provided 
by  the  user  session  results,  we  selected  a  set  of  historical  data  to  select 
inputs  for  Zeus  forecasts.  The  Zeus  historical  evaluation  used  surface 
synoptic  observational  data  obtained  from  the  National  Climatic  Data  Center 
(NCDC)  for  Dover  AFB,  Seymour  Johnson  AFB,  and  Fort  Bragg.  The  data  were 
selected  by  NCDC  from  the  U.S.  Air  Force  DATSAV  Station  files  and  recorded  on 
tape  in  the  TD  9685  format.  These  data  include  hourly  observations  of  many 
key  variables  required  by  Zeus  including  temperature,  dewpoint,  windspeed  and 
direction,  sky  cover,  and  visibility  at  all  three  bases.  Summary  of  Constant 
Pressure  Data  (WBAN)  and  Local  Climatological  Data  (LCD)  monthly  summaries 


provided  by  the  National  Climatic  Center,  contained  the  necessary  upper  air 
and  surface  data  to  complete  Zeus  input  tables.  Daily  weather  maps  published 
in  a  weekly  series  by  NOAA  were  used  to  determine  positions  and  motion  of 
surface  weather  systems. 

Using  data  provided  by  the  above  sources,  Zeus  runs  were  made  at 
12  and  00Z  for  selected  months  in  1980.  The  months  chosen  (January,  April, 
July,  and  October)  were  selected  to  ensure  that  each  season  was  represented 
in  the  evaluation.  Due  to  time  constraints,  only  the  first  2  weeks  of  each 
month  were  evaluated.  Additionally,  because  Zeus  does  not  provide  advice  on 
low  visibility  caused  by  precipitation,  all  runs  that  occurred  on  days  of 
restricted  visibility  due  to  rain  or  snow  were  omitted  from  the  tabulations. 
Official  National  Weather  Service  forecasts  (FOUS)  of  wind  direction  and 
speed  at  6,  12,  18,  and  24  h,  a  required  Zeus  input,  were  not  available.  To 
overcome  this  absence,  observed  surface  windspeeds  and  directions  were 
averaged  for  the  3  h  centered  on  the  6,  12,  18,  and  24  h  FOUS  times.  Tempera¬ 
ture  and  dewpoint  from  local  observing  stations  were  obtained  from  LCDs  for 
as  many  stations  as  were  available.  Because  some  stations  were  not  available 
and  frequently  the  required  hour  was  missing,  the  average  temperature-dewpoint 
differences  from  all  surrounding  available  stations  was  used  as  the  difference 
for  all  stations.  The  stations  used  for  Zeus  data  for  each  airbase  forecast 
are  listed  in  TABLE  28.  Because  synoptic  weather  charts  were  available  daily 
at  12Z,  it  was  necessary  to  interpolate  between  charts  to  obtain  reasonable 
positions  of  weather  systems  at  00Z.  Dover  AFB  inputs  of  850  mbar  windspeed 
and  direction  were  not  available  from  Atlantic  City,  New  Jersey,  prior  to 
September  1980.  These  inputs  were  substituted  with  850  mbar  data  observed  at 
Fort  Totten,  New  York. 


TABLE  28.  Local  Stations  Used  to  Obtain  Temperature  and  Dewpoint 
Data  from  Local  Climatological  Data  Summaries. 


Location 


Local  Temperature-Dewpoint  Observations 


Dover  Atlantic  City,  New  Jersey 

Baltimore  International  Airport,  Maryland 
Wallops  Island,  Virginia  (12Z  only) 
Wilmington,  Delaware 

Fort  Bragg  Cape  Hatteras,  North  Carolina 

Norfolk,  North  Carolina 
Seymour  Johnson  AFB,  North  Carolina 
Wilmington,  Delaware 

Cape  Hatteras,  North  Carolina 
Fort  Bragg,  North  Carolina 
Norfolk,  North  Carolina 
Wilmington,  North  Carolina 


Seymour  Johnson 
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Contingency  tables  of  forecasts  and  observations  of  three  visibility 
categories  at  Fort  Bragg  are  presented  for  the  months  of  January,  April, 

July,  and  October  and  for  all  periods  combined  in  TABLE  29.  The  same 
sets  of  three-by- three  contingency  tables  are  presented  in  TABLES  30  and 
31  for  Seymour  Johnson  and  Oover,  respectively.  An  overall  set  of 
contingency  tables  for  the  three  bases  combined  is  presented  in  TABLE  32. 

Cases  of  rain  or  no  forecast  by  Zeus  during  the  periods  of  evaluation  are  not 
included  in  these  tables. 

As  can  be  seen  in  all  the  contingency  tables,  the  occurrence  and 
forecast  of  category  3  is  the  most  conrion  contingency,  occurring  88  percent 
of  the  time  or  247  out  of  280  cases.  The  remaining  33  cases  include 
31  observations  of  category  1  or  2  visibility  and  15  forecasts  of  category  1 
or  2  visibility.  This  suggests  that  the  frequency  of  forecast  of  category  1 
and  2  is  too  low  by  a  factor  of  2.  For  the  forecasters  at  Raleigh  (see 
TABLE  26)  the  frequency  of  forecasting  category  1  and  2  exceeded  the  number 
of  occurrences.  This  showed  a  tendency  to  overforecast  fog  or  be  more 
conservative  in  issuing  low  visibility  forecasts.  Out  of  83  cases,  there 
were  30  category  1  or  2  forecasts  compared  to  24  observations.  Nine  (or 
30  percent)  of  the  30  Raleigh  forecasts  were  correct  while  6  (or  40  percent) 
of  the  15  Zeus  forecasts  in  TABLE  32  were  correct.  The  primary  lesson  here 
is  that  the  Zeus  system  may  need  some  further  refinement  to  increase  the 
frequency  with  which  low  visibility  advice  is  offered.  Although,  as  subsequent 
discussion  will  bring  out,  the  overall  accuracy  of  the  system  is  probably 
better  than  the  average  forecaster,  the  advice  on  low  visibility  forecasts 
may  be  the  most  important  advice  given. 

TABLE  33  presents  a  sumnary  of  skill  scores  and  P-scores  for 
each  of  the  contingency  tables  in  TABLES  29  to  32.  Over  all  locations 
and  periods  the  skill  score  was  0.38  and  the  P- score  was  0.18.  The  scores 
over  all  periods  for  each  of  the  three  bases  range  from  0.23  for  Fort  Bragg 
to  0.55  for  Dover  (skill  scores)  and  from  0.26  for  Fort  Bragg  to  0.12  for 
Dover  (P-score,  low  is  best).  The  scores  over  all  bases  for  each  of  the  four 
periods  range  from  -0.03  for  January  to  0.51  for  October  (skill  score)  and 
from  0.29  for  July  to  0.14  for  April  (P-score).  The  best  skill  scores  were 
obtained  for  the  October  period  at  each  base,  and  the  worst  skill  scores  were 
for  January.  This  is  true  partly  because  there  are  fewer  observations  and 
fewer  correct  forecasts  for  January  and  April  compared  to  July  and  October. 

This  is  not  because  poor  visibility  does  not  occur  in  January  and  April,  but 
because  the  poor  visibility  is  mostly  associated  with  precipitation  accompany¬ 
ing  winter  storms,  for  which  Zeus  is  not  designed  to  advise.  Cases  of  poor 
visibility  associated  with  precipitation  were  excluded  from  the  evaluation. 
However,  in  cases  where  radiation  and  advection  are  the  primary  causes  of 
fog,  the  system  does  show  skill  as  indicated  by  the  skill  scores  for  July  and 
October  at  all  bases  and  for  April  at  Dover.  The  skill  score  is  very  sensitive 
to  the  number  of  correct  forecasts  in  categories  with  low  observed  frequencies. 
Months  for  which  there  were  no  correct  forecasts  in  categories  1  and  2  all 
produced  low  skill  scores.  A  more  accurate  skill  score  requires  that  more 
cases  be  validated. 
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TABLE  29.  Contingencies  of  Forecast  and  Observed  Occurrences 
of  Three  Categories  of  Visibility  at  Fort  Bragg. 


a.  January 

Observed 


Forecast 


Forecast 


Forecast 


Observed 


Forecast 


Combined 
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b.  April 


d.  October 

Observed 

3  2 


Forecast 

Observed 

3 

2 

1 

3 

74 

8 

0 

2 

1 

0 

0 

1 

0 

2 

1 

TABLE  30.  Contingencies  of  Forecast  and  Observed  Occurrences 
of  Three  Categories  of  Visibility  at  Seymour  Johnson. 


a. 

January 

b.  April 

Forecast 

Observed 

Forecast 

Observed 

3 

2 

1 

3 

2 

1 

3 

17 

2 

0 

3 

22 

1 

0 

2 

0 

0 

0 

2 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

c. 

July 

d.  October 

Observed 

Observed 

Forecast 

3 

2 

1 

Forecast 

3 

2 

1 

3 

18 

1 

0 

3 

21 

2 

0 

2 

0 

0 

0 

2 

0 

0 

0 

1 

1 

2 

1 

1 

0 

1 

1 

e.  Combined 


TABLE  31.  Contingencies  of  Forecast  and  Observed  Occurrences 
of  Three  Categories  of  Visibility  at  Dover. 


a. 

January 

b.  April 

Forecast 

Observed 

Forecast 

Observed 

3 

2 

1 

3 

2 

1 

3 

24 

0 

0 

3 

22 

1 

0 

2 

0 

0 

0 

2 

0 

0 

0 

1 

0 

0 

0 

1 

0 

1 

1 

c. 

July 

d.  October 

Observed 

Observed 

Forecast 

3 

2 

1 

Forecast 

3 

2 

1 

3 

23 

1 

1 

3 

26 

1 

0 

2 

0 

0 

0 

2 

0 

1 

0 

1 

0 

1 

1 

1 

0 

0 

0 

e. 

Combined 

Observed 

Forecast 

3 

2 

1 

3 

95 

3 

1 

2 

0 

1 

0 

1 

0 

2 

2 
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TABLE  32.  Contingencies  of  Forecast  and  Observed  Occurrences 
of  Three  Categories  of  Visibility,  Combined  for  Three  Bases. 


a.  January 


b.  April 


Forecast 

Observed 

Forecast 

Observed 

3 

2 

1 

3 

2 

1 

3 

58 

4 

0 

3 

63 

4 

0 

2 

1 

0 

0 

2 

0 

0 

0 

1 

0 

0 

0 

1 

0 

1 

1 

c 

July 

( 

J.  October 

Observed 

Observed 

Forecast 

3 

2 

1 

Forecast 

3 

2 

1 

3 

58 

4 

1 

3 

68 

5 

0 

2 

0 

0 

0 

2 

0 

1 

0 

1 

1 

4 

2 

1 

0 

2 

2 

Combined 


Observed 


Forecast 


TABLE  33.  Skill  Scores  and  P-Scores  for  Zeus  Forecasts  on 
Historical  Data  by  Location,  Period,  and  Combined. 


Location 

Period 

Skill 

Score 

P-Score 

Number  of 
Cases 

Fort  Bragg 

January 

-0.07 

0.30 

20 

April 

0.00 

0.19 

21 

July 

0.22 

0.30 

20 

October 

0.46 

0.24 

25 

All 

0.23 

0.26 

86 

Seymour  Johnson 

January 

0.00 

0.21 

19 

Apri  1 

0.00 

0.09 

23 

July 

0.44 

0.34 

23 

October 

0.46 

0.24 

25 

All 

0.38 

0.22 

90 

Dover 

January 

_ * 

0.00 

24 

April 

0.57 

0.16 

25 

July 

0.46 

0.22 

27 

October 

0.65 

0.07 

28 

All 

0.55 

0.12 

104 

All 

January 

-0.03 

0.16 

63 

Apri  1 

0.36 

0.14 

69 

July 

0.40 

0.29 

70 

October 

0.51 

0.18 

78 

All 

0.38 

0.19 

280 

*  Undefined  because  only  one  category  was  forecast  and  observed. 
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In  general,  the  Zeus  system  shows  skill  in  forecasting  visibility 
as  measured  by  statistical  scores.  This  is  evident  by  the  overall  skill 
scores  of  0.38  on  the  historical  data  and  0.35  on  the  user  run  cases.  In 
addition,  the  Zeus  system  results  on  user  run  cases  showed  better  skill 
scores  than  forecasters  for  the  same  period  and  location.  The  Zeus  skill 
score  for  Seymour  Johnson  user  runs  was  0.46  compared  to  the  Seymour  Johnson 
forecaster  score  of  0.23  and  Raleigh,  North  Carolina,  forecaster  score  of 
0.24.  In  addition,  in  comparing  P-scores,  the  Zeus  system  showed  39  percent 
improvement  over  Seymour  Johnson  forecasters  and  63  percent  improvement  over 
Raleigh  forecasters. 

There  is  another  characteristic  of  the  Zeus  forecasts  worth  noting. 
When  the  results  from  the  historical  data  are  combined  with  the  user  runs 
with  Zeus  for  category  1  and  2  conditions,  we  find  that  Zeus  forecast  these 
categories  31  times  while  they  were  observed  59  times.  However,  the  Zeus 
forecast  was  correct  14  times,  or  24  percent  of  the  time.  Category  3  was 
forecast  36  times  when  1  or  2  was  observed  or  61  percent  of  the  time.  In  the 
remaining  nine  cases,  category  1  was  forecast  when  category  2  occurred.  The 
data  in  TABLES  25  and  26  show  that  forecasters  at  Seymour-Johnson  and  Raleigh 
forecast  the  lowest  two  categories  of  visibility  44  times  and  it  was  observed 
44  times.  They  were  correct  13  times,  or  30  percent  of  the  time.  Category  3 
was  forecast  23  times,  or  52  percent  of  the  time  when  category  1  or  2  was 
observed.  These  results  suggest  that  there  may  be  a  tendency  for  Zeus  to 
slightly  underforecast  the  frequency  of  occurrence  of  the  low  visibility 
cases. 


Finally,  the  number  of  cases  for  which  the  system  has  been  evaluated 
is  relatively  small.  The  results  do  not  allow  definitive  conclusions  to  be 
developed  about  performance  during  specific  times  of  the  year.  In  particular, 
the  results  for  performance  during  January  and  the  winter  period  is  indefinite 
because  of  the  small  number  of  low-visibility  cases  without  precipitation 
influence. 
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Section  6.0 

PROPOSED  TASKS  FOR  IMPROVING  AND  EXPANDING  THE  USE  OF  ZEUS  KBES 

FOR  AIRBASE  FORECASTERS 


As  a  result  of  initial  efforts  to  apply  artificial  intelligence 
techniques  to  visibility  forecasting,  it  is  clear  that  the  use  of  a  knowledge 
based  system  has  merit,  is  needed  and  wanted  by  operational  forecasters,  and 
can  be  a  useful  guide  to  inexperienced  forecasters. 

When  the  forecaster  is  busy  (often  due  to  changing  and  marginal 
weather  situations),  the  automated  weather  advisor  can  save  the  forecaster 
valuable  time.  Based  on  experience  gained  in  developing  an  initial  demon¬ 
stration  system,  the  following  are  suggested  ways  to  improve  the  usefulness 
of  the  initial  product  to  forecasters  at  airbases: 


•  Modify  and  extend  the  initial  system  based  on  detailed 
interviews  with  system  users  after  they  have  had 
several  months  of  experience  with  it  and  based  on 
statistical  evaluations  of  the  system's  performance. 

•  Develop  the  capability  of  the  system  to  accept  inputs 
continuously  from  online  data  interfaces;  extend  the 
system's  capability  to  aid  in  a  continuous  weather-watch 
mode  and  to  issue  warnings  to  forecasters  of  possible 
pending  changes 

•  Add  the  capability  to  advise  on  forecasting  cloud 
ceiling  height  to  the  present  system  capability. 

•  Develop  guidance  for  implementing  KBES  at  other  bases 
and  develop  command-level  plans  for  Air-Weather  - 
Service-wide  implementation. 


H 
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An  advisory  visibility  forecasting  system  has  been  developed  for 
each  of  three  airbases:  Dover  AFB,  Seymour  Johnson  AFB,  and  Fort  Bragg  AAF. 
The  KBES  for  each  base  contains  general  rules  that  are  applicable  to  any 
base,  and  local  rules  that  are  specific  to  a  single  base  or  a  group  of 
nearby  bases.  For  each  KBES,  the  user  is  advised  about  expected  future 
visibility  conditions  (up  to  12  h  in  advance)  as  a  result  of  answering  a 
series  of  questions  about  existing  and  recent  weather  or  about  readily 
available  synoptic  weather  analyses.  The  specific  questions  to  which  the 
user  must  reply  depend  on  his/her  answers  to  previous  questions.  In  the  end, 
the  user  is  advised  about  the  occurrence  of  different  levels  of  visibility 
for  forecast  periods.  The  user  may  Interrogate  the  system  to  determine  what 
rules  it  used  to  develop  its  advice.  In  addition,  the  user  may  examine 
“what  if"  propositions  to  determine  how  the  system's  advice  will  change  if 
different  input  information  is  available.  Differences  in  input  information 
may  be  selected  to  represent  uncertainties  in  the  information  provided.  The 
tasks  described  here  are  modifications  and  extensions  of  three  visibility 
KBESs  that  are  currently  under  development.  The  proposed  work  is  based  on 
lessons  learned  in  the  course  of  developing  these  initial  systems. 
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Section  7.0 


CONCLUSIONS 


This  study  has  analyzed  the  feasibility  of  using  the  techniques  of 
KBESs  from  the  field  of  AI  to  develop  a  system  to  advise  weather  forecasters 
on  local  airbase  visibility  conditions.  The  study  included  selecting  software, 
structuring  the  knowledge  base,  collecting  the  expertise,  developing  the 
advisory  system,  introducing  the  system  to  users,  and  evaluating  the  system. 
Three  versions  of  the  system  (Zeus)  were  developed  for  use  at  Dover  AFB,  Fort 
Bragg,  and  Seymour -John son  AFB.  The  following  conclusions  were  drawn  from 
the  study: 


1.  The  user  of  a  KBES  shell  that  runs  on  and  produces 
a  system  that  runs  on  a  PC-compatible  microscale 
computer  was  found  to  be  a  practical  way  to  develop  an 
advisory  system  for  weather  forecasters.  In  this 
study,  we  found  that  the  EXSYS  shell  allowed  us  to 
devote  a  minimal  amount  of  time  to  programming  and  to 
use  most  of  the  effort  to  collect  and  structure 
expertise  into  a  useful  knowledge  base. 

2.  After  reviewing  alternative  approaches,  we  found  that 
a  rule-based  system  that  uses  backward  chaining  with 
defined  goals  and  subgoals  was  the  best  approach  for  a 
KBES  to  advise  forecasters  on  visibility  conditions. 

3.  The  expertise  relevant  to  visibility  forecasting  was 
found  to  be  conveniently  divided  into  rules  relating 
to  advection  and  radiation  as  the  primary  physical 
processes  that  affect  visibility,  i.e.,  except  for 
precipitation  effects  that  were  excluded  early  in  the 
study.  Rules  relating  to  synoptic  considerations  were 
found  to  be  needed  as  a  related  knowledge  area.  Fog 
dissipation  processes  were  also  found  to  be  a  subject 


area  that  was  convenient  for  structuring  rules. 

Four  sources  of  expertise  found  to  be  useful  in  developing 
Zeus  were  the  open  literature,  Air  Weather  Service 
climatic  summaries,  local  airbase  technical  reference 
notebooks,  and  interviews  with  local  forecasters. 

Operational  considerations  found  to  be  important  to  users 
of  Zeus  were  the  time  required  to  interact  with  the 
system,  inconvenience  of  manually  entering  data, 
accuracy  of  the  advice,  friendly  prompting,  and  clear 
explanations  and  advice. 
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The  Zeus  system  was  found  to  be  an  excellent  way 
of  advising  forecasters,  based  on  the  favorable 
response  reported  by  users  in  80  percent  of  143 
uses . 

The  Zeus  system  was  found  to  give  skillful  advice 
to  forecasters  based  on  the  responses  on  user  survey 
forms  and  the  0.35  skill  score  computed  for  the  Zeus 
advice  given  in  158  uses. 

The  Zeus  system  was  found  to  give  accurate  advice 
when  given  historical  data  for  input,  based  on  results 
for  the  first  2  weeks  of  January,  April,  July,  and 
October  of  1980  for  each  of  the  three  bases.  The 
overall  skill  score  was  found  to  be  0.38. 

The  system  was  found  to  need  further  refinement  of 
its  rules  to  increase  the  frequency  with  which 
low-visibility  categories  are  forecast. 

When  the  Zeus  system  advice  developed  during  user 
interactions  was  compared  to  published  forecasts  from 
Seymour  Johnson  and  Raleigh,  North  Carolina,  for  the 
same  period,  the  Zeus  system  was  found  to  give  more 
accurate  advice  based  on  better  skill  scores  and 
p-scores. 

Overall,  we  conclude  that  the  Zeus  system  demonstrated 
the  feasibility  of  using  the  KBES  approach  to  assist 
forecasters.  In  particular,  the  system  demonstrated 
user  acceptance,  accuracy,  and  suitability  for  the 
weather  detachment  environment. 
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Appendix  B 


EXAMPLE  INTERVIEW 


Mark  Stunder  (S) 

"Today  is  February  26,  1986,  and  this  is  Mark  Stunder.  The  time  is 
1312  and  we  are  here  at  Simmons  Army  Airfield  and  we  are  going  to  be  discussing 
Central  North  Carolina  meteorology  with  Mr.  Bob  Madison.  Mr.  Madison  is  a 
civilian  meteorologist  here  at  Simmons  Army  Airfield. 

"Could  you  just  take  a  second  and  go  through  your  background 
in  meteorology  and  the  Air  Weather  Service?" 

Mr.  Bob  Madison  (M) 

"In  1963,  I  completed  the  basic  observer's  course  and  came  to  Fort 
Bragg  as  an  observer  in  the  fall  of  1963,  and  stayed  in  the  Fort  Bragg 
area  until  the  fall  of  1965.  As  an  observer  I  went  to  Korea  and  returned  to 
Pope  Air  Force  Base,  North  Carolina,  as  an  observer  in  1966  until  1969.  I  went 
through  the  basic  forecasting  course  at  Chanute  Air  Force  Base  in  May  1969 
until  January  1970.  I  graduated  from  the  basic  forecasting  course  and  was 
assigned  to  Shaw  Air  Force  Base  in  South  Carolina  in  January  1970,  ... 

"I  spent  3  years  at  McGuire  and  returned  to  Fort  Bragg  as  Assistant 
Station  Chief  in  July  1979.  I  retired  here  in  March  1983.  I  just  came  back 
on  board  as  civilian  forecaster  in  December  1985." 

(S)  "If  you  add  up  all  those  years,  how  long  would  you  have  been  at 

Fort  Bragg,  in  terms  of  forecasting  and  total  air  weather  service  time?" 

(M)  "About  7  years." 


($)  "What  I  would  like  to  do  is  just  go  through  a  couple  of  general 

cases,  see  what  you  think  of  some  of  these  scenarios,  and  see  if  you  would 
use  any  rules  of  thumb." 

"The  first  instance  that  comes  to  mind,  not  only  in  this  area  but 
throughout  most  of  North  Carolina,  is  the  northeasterly  wind  flow,  where  you 
would  have  a  high  pressure  system  up  over  New  England  or  as  far  south  as 
Pennsylvania.  In  that  case  we  may  be  looking  at  low  visibility  with  an 
increase  of  moisture  or  some  low  stratus,  or  maybe  even  a  little  bit  of  fog 
or  maybe  a  little  bit  of  drizzle,  but  mostly  a  fog  situation  versus  stratus." 

(M)  "Yes,  there  seems  two  offshoots  of  that  case.  One  is  if  you  drop 

a  back  door  down  to  Virginia  and  the  second  one  is  just  the  good  old  high 
pressure  system  without  a  front  but  with  sufficient  overwater  trajectory." 

(S)  "That's  right.  So  could  you  comment  first  on  the  back  door  situation?" 

(M)  "The  back  door  you  are  talking  about  is  when  you  get  an  elongated 

Eastern  seaboard  ridge  and  a  front.  You  get  a  relatively  persistent  ridge  up 
and  down  the  Eastern  seaboard  on  the  lee  side  of  the  Appalachians.  Usually 
along  with  that  you  have  an  overwater  trajectory  advection.  Usually  it  is  a 
bit  of  a  problem.  If  you  had  overwater  flow  for  18  to  24  hours,  then  it's  a 
good  case  to  forecast  below  mins.  If  the  ridge  persisted  for  48  to 
72  hours,  you'd  have  one  to  three  mornings  (of  below  mins).  If  you  have 
100  miles  of  overwater  trajectory,  the  anticyclone  curvature  and  subsidence 
below  3000  ft  to  compact  all  that  moisture  into  the  lower  50  to  100  mbar  at 
the  atmosphere,  It  Is  a  difficult  forecast  situation." 

(S)  “If  faced  with  that  potential  backdoor  situation  right  now,  what 

would  you?" 


(M)  "Beginning  with  the  forecast  date,  forecast  it  down--but  first  you 

have  to  forecast  If  the  front  win  pass  FB6.  Use  acetate  to  determine  if  you 


have  any  kind  of  persistent  southerly  movement.  If  it  shows  it  is  moving 
south  at  about  10  kn  (they  don't  move  directly  south  very  fast  on  this  side 
of  the  mountains),  then  a  back  door  cold  front  will  sneak  in.  This  (FBG)  is 
about  as  far  south  as  they  go.  If  you  can  prog  it  past  you,  I  would  tend  to 
drop  the  visibility  way  down.  If  the  850  mbar  trough  is  very  broad  and  a 
northwest  flow  across  the  mountains,  the  I  would  tend  to  drop  that  front 
down  into  the  850  mbar  trough  right  down  to  the  surface  until  it  is  just 
about  parallel  to  the  flow  and  trough. 

"If  that  extrapolation  puts  the  front  past  you,  begin  to  look  if 
low  visibility  Is  persistent  on  the  north  side  of  the  front.  Are  low  or 
below  normal  conditions  already  there  or  is  It  reasonably  dry  with  the  low 
very  far  offshore  and  just  the  ridge  is  pushing  It  (the  front)  along?" 

(S)  "Ok,  but  If  the  FBG  trajectory  is  veering  around  to  give  flow  from, 

say,  a  Wallops  Island  direction  then  what?" 

(M)  "That  Is  a  typical  fog  situation  with  winds  between  030  to  060.  To 

forecast  it  down.  If  It  Is  not  down  here,  you  look  for  appearance  of  low- 
level  moisture  at  Rocky  Mount  (RWI),  look  at  the  temperature  and  dewpoint  at 
Rocky  Mount,  Wilson  area  (If  they  are  In  business),  watch  Winston  (INT), 
Raleigh  (RDU),  Goldsboro  (GSB)--see  if  there  is  any  significant  dewpoint 
deflection.  If  there  are,  then  try  to  determine  the  rate  of  increase;  a  good 
estimate  can  be  made  as  to  when  It  will  reach  here." 

(S)  "So,  what  would  be  a  typical  time  lag  before  the  ceiling  lowers  if 

the  front  does  go  through  here  and  stops  about  50  miles  south?" 
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(M)  "About  6  to  9  hours.  A  stratus  situation  needs  about  a  5-  to  7-kn 

wind;  with  a  fog  situation,  it  takes  much  longer.  It  will  take  18  to  24 

hours  to  set  up.  You  need  the  persistent  overwater  flow.  You  have  to  see  it 

developing  and  have  the  increase  in  low  level-moisture,  not  only  on  the 
surface  but  at  least  through  50  mbar.  It  is  not  an  easy  forecast  to  make  but 
is  easy  after  it  develops." 

($)  "You  have  just  indicated  that  there  are  two  phenomena  that  can 

occur  with  backdoors,  stratus,  and  fog.  Could  you  further  comment  on  the 
moisture  parameters  you  would  examine  for  stratus?" 

(M)  "Stratus  will  come  in  with  5  to  7  kn  of  surface  wind.  You  will  look 

at  the  sfc  pressure  gradient.  Anything  less  than  5  to  7  kn  will  be  a  fog  and 
stratus  combination  or  just  fog.  The  stronger  the  high  is,  the  more  likely 
moisture  advection  you  will  have,  particularly  in  the  2000-  to  3000-ft  ranges. 
Looking  at  Wallops  Island  or  Hatteras  soundings,  if  2000-  to  3000-ft  winds  are 
on  the  order  of  25  to  30  kn  northeasterly,  you  look  for  advected  Atlantic 
stratus.  If  it  is  appearing  along  the  coastal  areas,  then  chances  are  it  will 
arrive  here.  If  the  cold  front  is  already  south  or  if  it  is  very  close  to 
you,  you  will  probably  still  have  low  ceiling  and  low  visibility  situation. 


The  position  of  the  front  is  very  important.  Usually  when  a  front  back  doors 
then  it  usually  already  has  the  ominipresent  moisture.  It  is  usually  socked- 
in  on  the  north  side  of  us  then  it  just  gradually  creeps  along.  It  is  a 
matter  of  timing;  if  the  front  is  through,  then  it  is  a  question  of  when  it 
is  going  to  go  away.. ." 

(S)  "The  major  back-door  case  then  appears  to  be  a  matter  of  timing  and 

whether  the  front  is  going  to  make  it  through.  But  *rfiat  about  a  high  pressure 
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system  moving  slowly,  to  your  north,  without  a  front--strong  high,  persistent 
flow?" 


3 
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"This  is  a  low  visibility  situation  that  will  occur  on  more  than 


one  morning,  usually  look  for  some  type  of  upslope  effect  in  western  Carolina. 


It  (air)  reaches  saturation  by  moving  gradually  uphill;  dewpoints  are 


relatively  high  but  there  is  a  6-,  8-  or  10-degree  spread  here,  but  by  the 


time  it  (air)  gets  out  to  the  Charlotte  area  it  is  saturated..." 


"How  would  you  utilize  LFM/FOUS  data  in  fog  forecasting?" 


"I  would  begin  to  look  to  fog  forecasting  if  I  had  greater  than 


50  percent  RH  in  R1  and  boundary- layer  winds  030  to  060  F0US--was  just 


civilian  stations  but  now  includes  military  stations  so  we  can  use  POPE." 


"The  LFM  has  become  so  “rel i able"  we  have  become  mental  cripples 


without  it." 


"When  do  you  bring  the  LFM  into  the  forecast  process?" 


"It's  immediate  with  most  forecasters." 


"But  how  does  Mr.  Madison  go  through  a  typical  forecast  using  the 


LFM?" 


"The  first  thing  I  do  is  get  a  cup  of  coffee.  I  start  out  by 


looking  at  the  surface  analysi s--take  an  acetate  and  get  12  to  24  hours  of 


continuity  on  the  pressure  centers  and  ridge  lines  and  positions.  We  (Fort 


Bragg)  do  a  local  area  0600Z  local  area  work  chart,  analyze  the  0600Z, 


analyze  that  workchart,  then  00Z  upper  air  emphasizing  the  850  mbar  analysis. 


doing  frontal  analysis,  moisture  analysis,  temperature  analysis  advection 


patterns,  moisture  and  stacking  the  surface  frontal  systems  to  the  850.  Then 


the  700  and  500  mb  and  300  mb  levels." 
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"The  first  3  to  6  hours  of  forecast  preparation  I  use  extrapolation 
techniques  which  work  very  well.  I  then  use  the  LFM  in  forecast  preparation 
in  the  12-  to  24-hour  period  for  downstream  for  system  development.  I  want  to 
see  what  the  LFM  package  does  with  the  trajectory  of  the  upper  air,  if  the 
LFM  is  bringing  it  through,  from  what  direction  etc.  I  look  at  the  relative 
humidity  package,  that's  a  good  surface  indicator  of  moisture." 
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